Analysis Software

From Jimenez Group Wiki
Revision as of 10:13, 20 November 2013 by Mlechner2 (talk | contribs) (→‎Real Time Software)
Jump to: navigation, search

If you would like access to the software on this page we ask that you adhere to a few basic conditions:

  • You don't share it further than your group, but direct any additional requests to us
  • We will answer simple questions and try to add requested features and especially fix bugs, but we can't "support" the software as if it were a commercially available package, we also will not answer basic Igor questions.
  • If eventually your group makes improvements, we'd like to have those shared back and ideally keep a master version.

All files on this page require proper credentials for download. Please contact us if you would like access to the files or to report any bugs with the software: password request

ICARTT

ICARTT Igor Software

Software Version Notes Link
ICARTT Loader and File Creation Tool 2013 Donna Sueper and Ken Aikin

ICARTT Documentation

 2013 General Macros (.ipf) 02.12.2013
2013 ICARTT Format (.ipf) 11.20.2013




ICARTT Igor Software Notes

  • These functions create and load data files of the format indicated in the 2008 ICARTT Data Management Plan document, which was roughly based on the Haines-Hipskind NASA Ames Format. Functions and macros herein are not appropriate for all data generated by ICARTT participants; assumptions are outlined below.
  • Functions in this file use functions in the 2008 GeneralMacros (latest version) igor procedure file. Users should have both files open in an igor experiment.
  • Whenever a user loads or creates a ICARTT format file, an igor data folder called ICARTTFileFormat gets created in the experiment and most of the work is done here. If something goes awry, the user can reset the data folder to root (most everyone's default)by inserting the text below (less the //) into the command line SetDataFolder root:
  • Some parameters are calculated; some have simple default values. They are:
    • NumHeaderLines - gets calculated
    • FileFormatType = 1001 // this value is assumed for reading and creating files.
    • FileVolumeNumber = 1 // this value is assumed for reading and creating files.
    • TotalVolumeNumber = 1 // this value is assumed for reading and creating files.
    • UTCDataStart - set from selected time wave
    • NumVariables - gets set when user selects waves
    • ScaleFactor = 1 // this value is assumed for reading and creating files.
    • MissingDataIndicator = -9999 // NOTE - The same missing value is used for all parameters when creating a file.
    • NumSpecialCommentLines - gets calculated
    • NumNormalCommentLines - gets calculated
    • Normal Comments are the concatenation of several other parameters
  • None of the original waves in the experiment are modified in any way. Waves selected to be saved are duplicated in the ICARTTFileFormat data folder. The UTC timewave (the independent parameter) and any dependent parameters formatted with a "t" (from the create panel) will have the starting date subtracted from it (in the created file) and will be formatted with one digit after the decimal.
  • When loading an icartt file into an experiment, the missing value as reported in the header will be used to convert missing values in the incoming waves to nans. The independent variable (time) will have the UTC date (as indicated in the header) added to it.
  • When loading data from the LoadDataOnly menu item, almost no format checking is done and missing data isn't changed to nans.
  • The name of the wave in the experiment will be the name given to the dependent and independent parameters, with the exception of matricies, where column names much as mymatrix0, mymatrix1, etc are used.
  • If anything is named in the experiment in a 'liberal' fashion (i.e. wave names with spaces), all bets are off.
  • Users should be aware that there are opearting system differences for end of line designations. Mac uses carriage return (CR), Unix uses line feed (LF), and Windows use carriage return + line feed (CRLF) For icarrtt, the preferred end of line designation is CRLF (which correspondes to \r\n in the code).

Other Igor ICARTT Software

The first tool was written by James Allan and uses one ipf:

ICARTT Matlab Software

ICARTT Python Software

  • Provided by: Barron Henderson barronh@gmail.com (21-Aug-2013)
  • ICARTT reader / netcdf converter for python
  • Part of PseudoNetCDF package found here

# In a python script
from PseudoNetCDF.icarttfiles import ffi1001
f = ffi1001('pathtofile')
var = f.variables['HO2_pptv']
print var[:].mean(), var.units

# From command line: dump CDF (like ncdump)
python -m PseudoNetCDF.pncdump -f ffi1001 pathtofile

# From command line: make a NetCDF file (like ncdump file1 | ncgen file2)
python -m PseudoNetCDF.pncgen -f ffi1001 pathtofile
pathtonetcdfoutput.nc

Picarro G2401

Picarro Software

Software Version Notes Link
Picarro G2401 Loader and Data Plotting v1.0 First release by Michael Lechner Picarro Panel (.ipf)

Picarro Software Notes

  • Picarro Panel v1.0 (MJL 02/05/2013)
    • Recursively loads all .DAT Picarro acquisition files or a specified date range of files in a given directory (best to point the loading directory to DataLog_User and leave the sub directories in their original configuration)
    • Plots:
      • Picarro measurement time series
        • CH4 / CH4 dry
        • CO
        • CO2 / CO2 dry
        • H2O
      • Picarro diagnostic time series
        • Das Temperature
        • Etalon Temperature
        • Warm Box Temperature
        • Cavity Temperature
        • Cavity Pressure

Software Output

Measurement Timeseries
Picarro Timeseries.jpg
Diagnostic Timeseries
Picarro Diagnostics.jpg

Real Time AMS Igor Display

Real Time Software

Software Version Notes Link
Real Time AMS Igor Display v10 First release Donna Sueper and Mike Cubison
Modified by Michael Lechner 		DAQ_QuickView.ipf

Real Time Notes

  • THIS FUNCTION WILL NOT COMPILE WITHOUT APPROPRIATE XOP!
    • An XOP (external operation) is a modular chunk of code that adds features to Igor One needs the file called VDT2.xop or a shortcut to this file to be placed in the directory Wavemetrics:Igor Pro Folder:Igor Extensions: The VDT2.xop is normally found in the folder called Wavemetrics:Igor Pro Folder:More Extensions:Data Acquisition
  • Real Time (RT) and Serial Feed (SF) ipf
    • This code is for use with the ToF AMS data acquisition program. In the daq one has the option of outputting a limited set of parameters to one text file (TimeTrace.txt)many individual text files (RT_xxxxx.txt) and/or serial output (No file name, just a an old fashioned serial RS232 output. Most will only work with only the RT_xxx.txt files
    • The functions in this ipf are intended to run in an Igor experiment concurrently with the DAQ. The RT_xxxx.txt files generated by the DAQ are intended to be read into an Igor experiment on the DAQ computer. Absolutely nothing in this ipf should change anything that happens in the DAQ
  • Usage Instructions:
    • Aircraft / Ground Plot will bring up a plot of predetermined parameters focused on Aircraft / Ground data
    • Delete Real Time Files checkbox when unchecked files are read into igor and then the file is moved to the processed folder. When checked file is not moved to the process folder, but instead is deleted.
    • The "Clear.." button will clear existing waves - will set them to zero points. "Pause.." will resume appending values to existing waves.
    • The "Process MS itx files" checkbox causes the itx mass spectra files to be loaded and appended to a 2D m/z vs time wave (assuming they have been selected to be saved in the DAQ menu). No plotting or further averaging is done in the DAQ_QuickView ipf posted here.
  • The TimeTrace.txt nd the serial feed functions are intended to more specialized purposes.
    • The TimeTrace.txt file is intended to be read into an Igor experiment as one file, and no special code is needed (beyond that of creating an igor-friendly time wave). In this case, just create the Real Time panel and press the begin button.
    • The serial feed functions are intended to work on a computer that has a serial stream connection to the daq. In this case, just create the Serial Feed panel and press the begin button. In either case, a standard graph of values will be plotted. Users may make new plots or change the existing one to suit their needs.

Software Output

Aircraft Data Plot
Real Time Plot.jpg

SMPS

SMPS Software

Software Version Notes Link
SMPS data loader v4 First release Doug Day (built on MJC's original loader) JG_SMPS_v4.pxt
Example experiment
Sample AIM exported text files

SMPS Software Notes

JG_SMPS_V4 (dd 1/12/11)

  • Loads and Concatenates SMPS data collected with TSI AIM (built on Mike C's BEARPEX/CalNex JG_SMPS_v3.ipf and uses ToolsMJC_v1a.ipf and Tools_20100221.ipf)
  • Input should be .txt files created as ouput in TSI AIM software (row-formatted) and dNdLogDp and in named in format: 2010_12_13_1522_SMPS.txt (yyyy_mm_dd_hhmm_SMPS.txt)
  • "Load-Concatenate" button promps user to select folderpath for text files to load

and also promps user to input assign "project" name to files loaded (will put all data in folder names as such

  • Image Time Series Plots plots either number or mass distributions (density approximated as 1.0 g/cm3)

Optional inputs for: Interpolation across data gaps Dp smoothing Max percentile of data assigned to hottest color in image (help use full color range)

  • Integrated # and Mass Concentration Plot (Number or Mass)

Optional inputs: Max/Min Dp over which to integrate total concentration

Additional Notes:

  • Best to reselect "Data Folder to Plot" to be sure that datafolder is set to proper project folder (should be reset after all plotting, so only really needed if user changes datafolders). However, must select before first plot.
  • Panel inputs stored in root:Temporary (DO NOT DELETE this folder) - can delete all other folders
  • Scaled dXdlogDp waves: Currently, these are not used anywhere in the code. To avoid make large "scaled waves" (interpolated to new/fixed timestamp) see the line near the top of the main ipf: "Variable offset = date2secs(2010,12,13), delta=60 //DD". As written (in the posted version) this would make a scaled size distribution from Dec, 13, 2010 with a point every 60 seconds up until YOUR last scan time. To avoid make this long, unused, mostly empty wave, change the date to the day before you collected your data and maybe change the delta to something bigger, like 300+. Then those waves will be small. Will code with more elegant solution in future version.

Next Version Steps:

  • some QC functions like in CalNex example
  • option for mask waves

Software Output

dNdlogDp Size Distribution
Dn sizedist.jpg
dMdlogDp Size Distribution
Dm sizedist.jpg
dN Total Concentration
Dn total conc.jpg
dM Total Concentration
Dm total conc.jpg

Retrieved from "http://cires1.colorado.edu/jimenez-group/wiki/index.php?title=Analysis_Software&oldid=12817"