********************************************************* ********************************************************* Hierarchical Data from Igor HieDI 1.2 Jimenez Lab, University of Colorado at Boulder Released 11-December-2005 ********************************************************* ********************************************************* ******************** NOTES ON CURRENT RELEASE ******************** HieDI 1.2 is capable of converting data acquired with all versions of the ToF-AMS Data Acquisition Software. This includes: - Files that do not have dates in their filenames (~ pre May 2005) - Runs for which the info waves are included in the MS and PToF.itx files, and info.itx does not exist (upcoming ToF-AMS Acquisition software release 1.5). Note that data without info waves in info.itx, MS.itx, or PToF.itx cannot be converted When defining first and last runs, the user may select any type of file (info.itx, MS.itx, PToF.itx, or PToF.bin) All data files within the selected range must have the same filename structure. Files of type "YYMMDD_Run_Type.itx" and "Run_Type.itx" cannot be simultaneously converted. HDF files produced from older data may not be ready for SQUIRREL. For instance, certain data sets do not have parVal waves and others have parVal waves that lack critical values. Processing of these data sets may require custom changes in SQUIRREL. A button has been added that allows the user to stop a conversion mid-way, before the final run is reached. HDF files for all runs processed prior to the stop event will be complete and ready for SQUIRREL processing. Stopping requires double-clicking the button. If the length of any array (eg MSopen or PToF_sticks) changes between runs, a new HDF file is started. This applies to c-ToF data sets where, for instance, the pulser period is changed and to hr-ToF data sets that alternate between v and w configurations. Note, that in the real-time data sets a single HDF file can contain both v and w data. InfoStr and ParStr are written as attributes of the datasets InfoVal and ParVal. These strings are read from the files being converted. The dataset RunInfo has attributes describing the (7) values in the dataset. This release includes logic for situations where the first run of the HDF file has MSOpen and/or MSclosed and subsequent runs do not have these waves. Layers of corresponding datasets are filled with NaN if no data exists. Will ignore *_MS.itx or *_PToF.itx that contain info waves but no data. ******************** GENERAL ******************** This program converts .itx and .bin data files acquired on the Aerodyne Time-of-Flight Mass Spectrometer to Hierarchical Data Format (HDF). The format of the generated HDF files is identical to that of the HDF files that will be written in real-time from ToF-AMS Acquisition software versions 1.5 and later. A single HDF file contains many runs. Data within the file is sorted into datasets, such as "InfoVal," "ParVal," and "MSopen." The layers of these datasets are the individual runs. All HDF data can be viewed and processed using the HDF5XOP available with IgorPro 5.04 ******************** INSTALLATION ******************** The .net framework must be installed before running HDFwriter. Note that if you are running the ToF-AMS data analysis software, the framework is already installed. If not, the framework is available at: http://msdn.microsoft.com/netframework/default.aspx HieDI_vXXX.exe should be saved to the same directory as ToF_AMS_vXXX.exe files, which is typically: C:\Program Files\ToFAMS\ToFAMSAcq HieDI requires a set of dlls, which should be saved to the same directory as HieDI_vXXX.exe. These can be downloaded from: http://cires.colorado.edu/jimenez-group/ToFAMSResources/ToFSoftware/Downloads/Supplemental/Required_HDF_dlls.zip ******************** USE ******************** The set of .itx and .bin files to be converted is chosen by selecting data files corresponding to the first and last runs of interest. Run numbers within the desired set of files do not need to be contiguous and the set of files can cover multiple dates. But, info waves must exist for all runs to be converted. PToF.bin cannot be accurately converted without an associated PToF.itx file The status of the conversion process is tracked with labels indicating the current HDF File, the current Run Number being converted, and the total sizes of the Main and PToF HDF files. A given HDF file will be written to until either the PToF file size exceeds the maximum file size (defined in the "options" tab), the acquisition date changes, or the dimensions of a dataset within the file changes between runs (e.g., the MS Sticks array grows or shrinks). The destination directory for converted HDF data is defined in the "options" tab. The sub-directories \Main and \PToF will automatically be created within the destination directory. Created HDF file names include the acquisition date, the first run number written to the file (as 6 digits), and the type of data in the file (p = ptof, m = main). As an example, writing to the destination directory C:\ToFAMS\ToFAMSData\HDFConvertData and starting the conversion with file 103105_100_Info.itx will generate C:\ToFAMS\ToFAMSData\HDFConvertData\Main\103105_000100_m.hdf C:\ToFAMS\ToFAMSData\HDFConvertData\PToF\103105_000100_p.hdf _m.hdf files include the datasets: InfoVal,ParVal, RunInfo, MSSDiff, MSDiff, MSopen, and ptof_stick _p.hdf files include the datasets: InfoVal,ParVal, RunInfo, and PToF Datasets MSSDiff, MSDiff, MSopen, ptof_stick, and PToF are given extensions to indicate the ToF type. - No extension corresponds to c-ToF - "_v" corresponds to v-ToF - "_w" corresponds to w-ToF An HDF file may contain both _v and _w datasets The dataset RunInfo contains rows describing the acquisition runs written to the extendible data sets. - Column 1 contains the Run Number - Column 2 contains the Year - Column 3 contains the Julian Day - Column 4 contains the seconds after midnight - Column 5 contains the ToF type (0=c, 1=v, 2=w; For older data, where ToF type was not included in ParVal, HieDI will output NaN) - Column 6 contains a value representing the count of the defined ToF type in the current HDF File (used for Squirrel Indexing