Analysis Software

From Jimenez Group Wiki
Revision as of 01:39, 11 September 2014 by Zhe Peng (talk | contribs) (→‎Input Mechanisms)
Jump to: navigation, search

If you would like access to the software on this page we ask that you adhere to a few basic conditions:

  • You don't share it further than your group, but direct any additional requests to us
  • We will answer simple questions and try to add requested features and especially fix bugs, but we can't "support" the software as if it were a commercially available package, we also will not answer basic Igor questions.
  • If eventually your group makes improvements, we'd like to have those shared back and ideally keep a master version.

All files on this page require proper credentials for download. Please contact us if you would like access to the files or to report any bugs with the software: password request

There is a google doc named JG_Igor_Tools that lists functions and brief descriptions in several ipfs.


ICARTT

ICARTT Igor Software

Software Version Notes Link
ICARTT Loader and File Creation Tool 2013 Donna Sueper and Ken Aikin

ICARTT Documentation

 2013 General Macros (.ipf) 02.12.2013 (version 1.03)
2014 ICARTT Format (.ipf) 05.28.2014 (version 2.13)




ICARTT Igor Software Notes

  • These functions create and load data files of the format indicated in the 2008 ICARTT Data Management Plan document, which was roughly based on the Haines-Hipskind NASA Ames Format. Functions and macros herein are not appropriate for all data generated by ICARTT participants; assumptions are outlined below.
  • Functions in this file use functions in the 2008 GeneralMacros (latest version) igor procedure file. Users should have both files open in an igor experiment.
  • Whenever a user loads or creates a ICARTT format file, an igor data folder called ICARTTFileFormat gets created in the experiment and most of the work is done here. If something goes awry, the user can reset the data folder to root (most everyone's default)by inserting the text below (less the //) into the command line SetDataFolder root:
  • Some parameters are calculated; some have simple default values. They are:
    • NumHeaderLines - gets calculated
    • FileFormatType = 1001 // this value is assumed for reading and creating files.
    • FileVolumeNumber = 1 // this value is assumed for reading and creating files.
    • TotalVolumeNumber = 1 // this value is assumed for reading and creating files.
    • UTCDataStart - set from selected time wave
    • NumVariables - gets set when user selects waves
    • ScaleFactor = 1 // this value is assumed for reading and creating files.
    • MissingDataIndicator = -9999 // NOTE - The same missing value is used for all parameters when creating a file.
    • NumSpecialCommentLines - gets calculated
    • NumNormalCommentLines - gets calculated
    • Normal Comments are the concatenation of several other parameters
  • None of the original waves in the experiment are modified in any way. Waves selected to be saved are duplicated in the ICARTTFileFormat data folder. The UTC timewave (the independent parameter) and any dependent parameters formatted with a "t" (from the create panel) will have the starting date subtracted from it (in the created file) and will be formatted with one digit after the decimal.
  • When loading an icartt file into an experiment, the missing value as reported in the header will be used to convert missing values in the incoming waves to nans. The independent variable (time) will have the UTC date (as indicated in the header) added to it.
  • When loading data from the LoadDataOnly menu item, almost no format checking is done and missing data isn't changed to nans.
  • The name of the wave in the experiment will be the name given to the dependent and independent parameters, with the exception of matricies, where column names much as mymatrix0, mymatrix1, etc are used.
  • If anything is named in the experiment in a 'liberal' fashion (i.e. wave names with spaces), all bets are off.
  • Users should be aware that there are opearting system differences for end of line designations. Mac uses carriage return (CR), Unix uses line feed (LF), and Windows use carriage return + line feed (CRLF) For icarrtt, the preferred end of line designation is CRLF (which correspondes to \r\n in the code).

Other Igor ICARTT Software

The first tool was written by James Allan and uses one ipf:

ICARTT Matlab Software

ICARTT Python Software

  • Provided by: Barron Henderson barronh@gmail.com (21-Aug-2013)
  • ICARTT reader / netcdf converter for python
  • Part of PseudoNetCDF package found here

# In a python script
from PseudoNetCDF.icarttfiles import ffi1001
f = ffi1001('pathtofile')
var = f.variables['HO2_pptv']
print var[:].mean(), var.units

# From command line: dump CDF (like ncdump)
python -m PseudoNetCDF.pncdump -f ffi1001 pathtofile

# From command line: make a NetCDF file (like ncdump file1 | ncgen file2)
python -m PseudoNetCDF.pncgen -f ffi1001 pathtofile
pathtonetcdfoutput.nc

Picarro G2401

Picarro Software

Software Version Notes Link
Picarro G2401 Loader and Data Plotting v1.0 First release by Michael Lechner Picarro Panel (.ipf)

Picarro Software Notes

  • Picarro Panel v1.0 (MJL 02/05/2013)
    • Recursively loads all .DAT Picarro acquisition files or a specified date range of files in a given directory (best to point the loading directory to DataLog_User and leave the sub directories in their original configuration)
    • Plots:
      • Picarro measurement time series
        • CH4 / CH4 dry
        • CO
        • CO2 / CO2 dry
        • H2O
      • Picarro diagnostic time series
        • Das Temperature
        • Etalon Temperature
        • Warm Box Temperature
        • Cavity Temperature
        • Cavity Pressure

Software Output

Measurement Timeseries
Picarro Timeseries.jpg
Diagnostic Timeseries
Picarro Diagnostics.jpg

Real Time AMS Igor Display

Real Time Software

Software Version Notes Link
Real Time AMS Igor Display v1.0 First release Donna Sueper and Mike Cubison
Modified by Michael Lechner 		DAQ_QuickView.ipf

Real Time Notes

  • THIS FUNCTION WILL NOT COMPILE WITHOUT APPROPRIATE XOP!
    • An XOP (external operation) is a modular chunk of code that adds features to Igor One needs the file called VDT2.xop or a shortcut to this file to be placed in the directory Wavemetrics:Igor Pro Folder:Igor Extensions: The VDT2.xop is normally found in the folder called Wavemetrics:Igor Pro Folder:More Extensions:Data Acquisition
  • Real Time (RT) and Serial Feed (SF) ipf
    • This code is for use with the ToF AMS data acquisition program. In the daq one has the option of outputting a limited set of parameters to one text file (TimeTrace.txt)many individual text files (RT_xxxxx.txt) and/or serial output (No file name, just a an old fashioned serial RS232 output. Most will only work with only the RT_xxx.txt files
    • The functions in this ipf are intended to run in an Igor experiment concurrently with the DAQ. The RT_xxxx.txt files generated by the DAQ are intended to be read into an Igor experiment on the DAQ computer. Absolutely nothing in this ipf should change anything that happens in the DAQ
  • Usage Instructions:
    • Aircraft / Ground Plot will bring up a plot of predetermined parameters focused on Aircraft / Ground data
    • Delete Real Time Files checkbox when unchecked files are read into igor and then the file is moved to the processed folder. When checked file is not moved to the process folder, but instead is deleted.
    • The "Clear.." button will clear existing waves - will set them to zero points. "Pause.." will resume appending values to existing waves.
    • The "Process MS itx files" checkbox causes the itx mass spectra files to be loaded and appended to a 2D m/z vs time wave (assuming they have been selected to be saved in the DAQ menu). No plotting or further averaging is done in the DAQ_QuickView ipf posted here.
  • The TimeTrace.txt nd the serial feed functions are intended to more specialized purposes.
    • The TimeTrace.txt file is intended to be read into an Igor experiment as one file, and no special code is needed (beyond that of creating an igor-friendly time wave). In this case, just create the Real Time panel and press the begin button.
    • The serial feed functions are intended to work on a computer that has a serial stream connection to the daq. In this case, just create the Serial Feed panel and press the begin button. In either case, a standard graph of values will be plotted. Users may make new plots or change the existing one to suit their needs.

Software Output

Aircraft Data Plot
Real Time Plot.jpg

SMPS

SMPS Software

The SMPS Software is generally not shared at this point, except with close collaborators of our group, as we don't have time to support it.

Software Version Notes Link
SMPS data loader/plotter v4 First release Doug Day (built on MJC's original loader) JG_SMPS_v4.pxt
Example experiment
Sample AIM exported text files
SMPS data loader/plotter v4.46
(latest)		 6 Aug. 2014
-Revised so that only supporting ipf needed is DS_GenTools.ipf (included in pxt posted here). See version notes in header of main .ipf 		JG_SMPS.ipf
JG_SMPS_v4_46.pxt

SMPS Software Notes

JG_SMPS_V4.46 (dd 8/06/14)

  • Loads and Concatenates SMPS data collected with TSI AIM (built on Mike C's BEARPEX/CalNex JG_SMPS_v3.ipf and used ToolsMJC_v1a.ipf and Tools_20100221.ipf)
  • As of V4.46 uses DS_GenTools.ipf for general functions (previous to then, used JG_GenTools or DougTools.ipf))
  • Input should be .txt files created as ouput in TSI AIM software (row-formatted) and dNdLogDp and in named in format: 2010_12_13_1522_SMPS.txt (yyyy_mm_dd_hhmm_SMPS.txt)
  • If starting with .ipf (rather than .pxt, .pxp);
    • 0) Compile
    • 1) Run function "PopulateSMPSpanel()" (which populates the panel with default input values in "root:Temporary")
    • 2)"SMPS Plotting Panel" from "SMPS" menu dropdown
    • 3) If using a mask wave, a) customize "AssignFlags()" function and run; b) load mask and time waves in "Masterlogger folder"
  • "Load-Concatenate" button prompts user to select folderpath for text files to load and also promps user to input assign "project" name to files loaded (will put all data in folder names as such)
    • Choose the version of TSI AIM used to export text files (diagnostic columns vary)
    • User can chose number of Dp bins to regrid dNdLogDp to at the loading step only. This is useful for when file of different Dp ranges are loaded together. (when plotting later can chose to use these or the native bins). Warning, uses a linear interpolation, not binning, so if much fewer bins than in original data is chosen,not all data points are used in regridding so statistics may be degraded. Thus best to chose similar Dp number to original data and use raw data for final workups, instrument comparison, etc.
  • Image Time Series Plots plots either number or volume distributions
    • Optional inputs for:
      • Dp smoothing
      • Max percentile of data assigned to hottest color in image (help use full color range)
      • Interpolation across data gaps (of user-specified duration)
      • Choosing whether to use regridded or raw dXdLogDp (native scaling)
  • Integrated # and Mass Concentration Plot (Number or Mass)
    • Optional inputs: Max/Min Dp over which to integrate total concentration

Additional Notes:

  • Best to reselect "Data Folder to Plot" to be sure that datafolder is set to proper project folder (should be reset after all plotting, so only really needed if user changes datafolders). However, must select before first plot.
  • Panel inputs stored in root:Temporary (DO NOT DELETE this folder) - can delete all other folders
  • Scaled dXdlogDp waves: Currently, these are not used anywhere in the code. To avoid make large "scaled waves" (interpolated to new/fixed timestamp) see the line near the top of the main ipf: "Variable offset = date2secs(2010,12,13), delta=60 //DD". As written (in the posted version) this would make a scaled size distribution from Dec, 13, 2010 with a point every 60 seconds up until YOUR last scan time. To avoid make this long, unused, mostly empty wave, change the date to the day before you collected your data and maybe change the delta to something bigger, like 300+. Then those waves will be small. Will code with more elegant solution in future version.
  • There are several auxillary functions at the bottom that may be useful for QC, exporting data. Use "CutBadDat()" function to "manually" enter bad periods that thereafter will be cut when using panel buttons.
  • Version update notes at top of .ipf

Next Version Steps:

  • Several. See top of ipf. Feel free to send suggestions to Doug.

Software Output

Panel v4.44
JG SMPS Panel v4 44.png
dNdlogDp Size Distribution
Dn sizedist.jpg
dMdlogDp Size Distribution
Dm sizedist.jpg
dN Total Concentration
Dn total conc.jpg
dM Total Concentration
Dm total conc.jpg

KinSim

KinSim Software

Software Version Notes Link
KinSim v2 First release by Harald Stark
6 Sep. 2014 Modified by Zhe Peng 		KinSim_v2.ipf

KinSim Software Notes

KinSim_v2.ipf (ZP 9/6/14)

  • Version 1 (by Harald Stark): mechanism generation and integration of rate equations.
  • Version 2: functions in v1 and
    • P- and T-dependent rate constants
    • Advanced input of initial concentrations - compatible to direct input of numbers (for concentrations in molec/cm3), and mixing ratios in "%", "ppth", "ppm", "ppb", "ppt", and "ppq"
    • Optimized parameters for the integrator - to handle stiff problems
  • General usage: compile the procedure then select "Display_help" from the "KinSim" menu. After the dialog box is prompted, select "Basic version" if only the functions in v1 are needed, else select "Version 2". Follow the help notes.

Input Mechanisms

  • Basic OFR mechanism (ZP 9/10/14)
  • Mechanism of CH4 full oxidation in OFRs (ZP 9/10/14)
  • Doug's chamber mechanism (Aug 2014)

Other Utilities

  • Doug's plot macros (Aug 2014)
Retrieved from "http://cires1.colorado.edu/jimenez-group/wiki/index.php?title=Analysis_Software&oldid=13519"