Difference between revisions of "Sq pk programming"

From Jimenez Group Wiki
Jump to: navigation, search
(Squirrel)
(To Do Priority)
 
(164 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
===To Do===
 
===To Do===
  
* Delphine wanted to spit out the elemental mass spectra without normalizing them all to average total mass.
+
* Add the ability to do EA on non HROrg species.
* Get and incorporate Jose's read me to the wiki
 
* Allow users to select HR spectra from non-root data folders
 
  
=== Done ===
+
* Add more web documentation, with specific examples of mass calculations.
  
* Ken had an issue when he chose the tag wave instead of the text wave as the chemical mass formula wave. Perhaps there should be a warning that appears when x number of mass fragments are flagged as having issues instead of simply printing them to history. (done in 1.02)
+
==Pika==
* Add a new table on the web page for downloading ipf (done in 1.01)
+
 
* Add ability to not require the fit of CO and/or CO2 (done in 1.01)
+
===To Do Priority===
* Add the ability to do the analysis on user selected HR spectra, such ans PMF results (done in 1.01)
+
 
 +
* Do some sort of smoothing for the peak shape when user uses different left/right parameters.
 +
* The HR ion mass calculator, Query tool seems to need some sort of initialization before it works.
 +
* Remove use of HR_specName_list because it is redundant
 +
* for PMF prep error panel:
 +
** Jose wants the running median in the time series of the error and noise times series for individual sticks.
 +
** Do the time series plots differently so that users don't have to change axes
 +
** Graphs for isotopes hide isotopes
 +
** In prepping for PMF, nan isotopes
 +
** Add spike removal similar to what Qi Zhang did?
 +
 
 +
* Ask J if we have volunteer for the keeping of the HR list
 +
* Generate NeighboringHRIonSignal new type of Pika error
 +
* Pika fitting graph -  make it so that the user can resize
 +
* Doug wants H2O, OH, O and CO to be colored as CO2 in mass spec Org family plots.
 +
 
 +
* Create an HR frag check panel
 +
 
 +
* Add ability to plot in time series closed baseline sticks.
  
 +
* Check that m/z calibration panel works if one run isn't done first.
  
==Pika==
+
* Add K, j41K to default list of ions to fit
 +
 
 +
* For the time-varying peak width, have a better errror if the user doesn't select an HR ion that was used.
 +
 
 +
* Remove N3 from default list. (Done?)
 +
 
 +
* Add code to require and check that HR ions be formatted in a certain way - i.e. disallow H7C5 and require C5H7.
  
===To Do Priority===
+
* Change code so that for todo waves fit with different pw and ps but with the same set of HR ions, one can nicely generate time series or average mass spectra.  The code should check that the set of HR ions are the same for different todo waves.  If they are the same, then one should be able to proceed.
  
Final Peak Shape
+
* Add code to check that all the HR ions within each frag table element is fit. (Note: this is hard to do because of the recursive nature of frag entries!)
* We may want to allow a 'smoothly varying' peak width.
 
  
HR Fragment choices
+
* Make a Pika-specific airbeam correction?
* Begin to implement the HR frag table.  W need to introduce the concept of species. For example CO2 would belong partially to an HR-Organic and partially to the HR_air species.  This also involves the use of an HR batch table, In this HR batch table users also their own RIE, CE.  Note that this is a similar but slightly different concept than the family groupings.
 
  
Misc
+
* Implement the "resolution" fitting function for the peak width panel as default (Pete's suggestion)
* Change default behavior so that we have two HR difference sticks one which is found by fitting the raw diff spectra (called HRDiff), and an OpenMinusClosed which is simply the open HR sticks - closed HR sticks.
 
** Change the default behavior from fitting difference spectra to only fitting open and closed.  James' point at the user's mtg is that we should always be looking at the HR_OpenSticks - HR_ClosedSticks.
 
** Jose, 4-Apr-09: disagree with not fitting the diff, I think we should keep fitting it as for some peaks it is much more useful. However we should also show the open-closed fits (not just sticks) in the PIKA window. User should be able disable fitting the diff, and to use the diff results, even if the default it to use the open-closed results.
 
  
* A plot of the Residual Errors vs Counting errors (for one m/z and for chosen species
+
* Allow including other families (such as SO, NO) in elemental analysis along with HROrg (Puneet & Doug)
  
* m/z calibration and masses - We need to subtract the mass of an electron from all our masses.
+
* Tim O made a good suggestion for instances where peaks are not fit (resulting in nans).  Perhaps one could compare the  squirrel signal at an integer m/z, and if the value is sufficiently small, (the raw spectra shows noise), all the peak height values could be set to 0.  
  
* Begin to automatically create uncertainty values generated from peak shape and/or m/z calibration knowledge.
+
* We may want to allow a 'smoothly varying' peak width.
  
* Generate 'Neil's graph' whereby we plot as sticks from zero amount of C and H mass as a function of number of Oxygens.  Do a similar thing for the number of C's (the amount of H and Os and a function of number of Cs).
+
* Change all peak shape candidates to have a 0 entry in the left- and right-most value so that 'peak-shape type baselines' dont' get extended beyond +- 7 PW values.
  
* Checkbox for truncating sticks to zero in stacked graphs.
+
* Fully fold in Mike Cubison's sensitivity analysis code.
  
 
*  Many parameters panel
 
*  Many parameters panel
Line 50: Line 68:
 
** (Jose) This could be done with a radio button on top of the graph that selects SR/AR (meaning single run / all runs)
 
** (Jose) This could be done with a radio button on top of the graph that selects SR/AR (meaning single run / all runs)
  
* We need to add camels, dromedaries, and vw "bug" ions to the lists. We need some notation for these ions, let me suggest vw28, vw23, d28, d32, and c28, c32, with the comment column describing what they are. It is important that those are lowercase letters, so people don't confuse these ions with vanadium (V), tungsten (W), and deuterium (D).
+
===To Do (Less Urgent)===
 +
 
 +
* In HR_peakHeights_gr, change the color of the residuals to be different from the red, orange, brown.  Also add squirrel integration regions.  Also, add a checkbox to add remove black lines between raw data points.
 +
 
 +
* Perhaps shorten the names of some of the new error estimates (Pete)
 +
 
 +
* For Pika, we need to incorporate some sort of algorithm that sets the minimum error to be 1 ion.
 +
 
 +
* In average mass spec plots (including the 5-graph panel and the elemental analysis) Doug wants a way to 'move' isotopic child HR ions to their parent ion (i.e. j13C would be moved and grouped with the HR ion C.)
 +
 
 +
* From the HR_peakHeights_gr, when adding a brand new mass to the table, add a note in the history.  Also add a message when adding and removing masses to fit?
 +
 
 +
* m/z calibration and masses - We need to subtract the mass of an electron from all our masses. Note from Donna:  I want to leave this decision up to the HR list committee (Puneet, Niall, Delphine)
 +
 
 +
* Generate 'Neil's graph' whereby we plot as sticks from zero amount of C and H mass as a function of number of Oxygens.  Do a similar thing for the number of C's (the amount of H and Os and a function of number of Cs).
 +
 
 +
* In PeakWidth/Shape panel, add ability to not do average PW fits using the std dev as weights
  
===To Do (Less Urgent)===
+
* In the Graph 5 summaries, include one 'bar' type graph showing the fraction of each family to the total. 
  
* Create other methods for the user to select masses to fit beyond the simple default option.
+
* Pika on PToF spectra (gasp!)
  
 
* There is a need to consolidate the three sets of m/z lists that exist into a single one. We currently have the list of m/z possibly used in fits,(1) ones actually selected for HR sticks (2) list of m/z used for m/z calibration, and (3) the list of isolated ions. Each list is a subset of the previous one, and we can work with a single big list, and a series of masks to select / deselect the various m/z for the different purposes.
 
* There is a need to consolidate the three sets of m/z lists that exist into a single one. We currently have the list of m/z possibly used in fits,(1) ones actually selected for HR sticks (2) list of m/z used for m/z calibration, and (3) the list of isolated ions. Each list is a subset of the previous one, and we can work with a single big list, and a series of masks to select / deselect the various m/z for the different purposes.
  
 
* If user only does the m/z calibration and does not play with the HR panel, the code calculating the HR sticks may go awry (maybe - I need to check).
 
* If user only does the m/z calibration and does not play with the HR panel, the code calculating the HR sticks may go awry (maybe - I need to check).
* Incorporate some of Mike Q's functions that deal with sq frag vs pika sticks (Donna: Mike Q's July 2008 email) Also think about NH4 measure vs predicted and RIE for NH4 (Donna: Jose's July 2008 email).
+
 
 +
* Think about NH4 measure vs predicted and RIE for NH4 (Donna: Jose's July 2008 email).
 +
 
 
* Make clear what limitations there are about the total number of masses that can be fit at any m/z value.
 
* Make clear what limitations there are about the total number of masses that can be fit at any m/z value.
 
  
 
===To Discuss===
 
===To Discuss===
  
* Change saving of the HR sicks matrix from masses that were selected to fit to all known masses.
+
* Errors for input to PMF.
  
 +
===Done===
  
=== Done in latest Beta release===
+
Done in 2010 Users Mtg release:
  
Download Pika 1.04O template experiment[http://cires.colorado.edu/jimenez-group/ToFAMSResources/ToFSoftware/Downloads/PIKA/IntermediateRelease/ToF_AMS_HRAnalysis_v1_04O.pxt ToF_AMS_HRAnalysis_v1_04O.pxt])
+
* Doug wants a more clear warning when masses not fit (most likely due to nonmonotonicity in peak shape) in one spectra or many.
  
Download Pika 1.04M ipfs [http://cires.colorado.edu/jimenez-group/ToFAMSResources/ToFSoftware/Downloads/PIKA/IntermediateRelease/Pika104O.zip Pika104O.zip])
+
* In Diurnal plots, x axis label is on y axis.
  
Download Pika 1.04M explanatory ppt [http://cires.colorado.edu/jimenez-group/ToFAMSResources/ToFSoftware/Downloads/PIKA/IntermediateRelease/Pika104M.ppt Pika104M.ppt])
+
* Remove any mention of "fitting with SQ baseline subtracted in the panel", and replace with "fitting on top of the SQ baseline" (Jose)
  
 +
* We need to add camels, dromedaries, and vw "bug" ions to the lists. We need some notation for these ions, let me suggest vw28, vw23, d28, d32, and c28, c32, with the comment column describing what they are. It is important that those are lowercase letters, so people don't confuse these ions with vanadium (V), tungsten (W), and deuterium (D).
  
* Allow identification of peak width parameters/peak shape for certain runs. Done in 1.04P
+
* Jose wants to add other types of errors for folks to plot/use. If the current list of errors is considered to be a,b,c,d, we also want sqrt(a^2+d^2) and sqrt(a^2+b^2+d^2).  
  
* Doug wants the green sticks in the HR_PeakHeights_gr to be colored by family. Done in 1.04P
+
* In HR_PeakHeights graph perhaps color the individual peak shapes to the colors of the families.
  
* Currently some CS species get lumped into the CH species.  make a new family?  Done 1.04P
+
* Begin to implement the HR frag table.  W need to introduce the concept of species. For example CO2 would belong partially to an HR-Organic and partially to the HR_air species.  This also involves the use of an HR batch table, In this HR batch table users also their own RIE, CE.  Note that this is a similar but slightly different concept than the family groupings.
  
* Changed default behavior so that all pika operations are done on the raw spectra with the squirrel-calculated baselines removed.   (1.04M)
+
* Change default behavior so that we have two HR difference sticks one which is found by fitting the raw diff spectra (called HRDiff), and an OpenMinusClosed which is simply the open HR sticks - closed HR sticks.
 +
** Change the default behavior from fitting difference spectra to only fitting open and closed.  James' point at the user's mtg is that we should always be looking at the HR_OpenSticks - HR_ClosedSticks.
 +
** Jose, 4-Apr-09: disagree with not fitting the diff, I think we should keep fitting it as for some peaks it is much more useful. However we should also show the open-closed fits (not just sticks) in the PIKA window. User should be able disable fitting the diff, and to use the diff results, even if the default it to use the open-closed results.
 +
 
 +
* The peak width/shape panel needs tweaks to be more user friendly. The specific changes have not yet been worked out.
  
* Overhauled Peak Width/Shape panel. Too many changes to mention! (1.04M)
+
* The list of completed programming tasks has been moved to [http://cires.colorado.edu/jimenez-group/wiki/index.php/Sq_Pk_Programming_Completed#Pika http://cires.colorado.edu/jimenez-group/wiki/index.php/Sq_Pk_Programming_Completed#Pika]
  
* In final peak shape graph add ability to:
+
=== Important Bugs fixed since last release===
** truncate at 3.5 sigma
 
** trim wings to ensure monotonicity
 
* In final peak shape algorithm remove/modify final peak shape smoothing and/or make it user editable
 
  
* In Peak shape panel the first run number of index should be put into “run number” field (it defaults to 0)
+
There was an annoying bug that resulted in not being able to choose > 1 mass fragment each for the left and right-side peak shape candidate.  The code was fine in 1.05 and using Igor version < 6.10. But in Igor 6.10 there was a compilation error, and the quick fix for Igor 6.10 caused this behavior.  You can download PK_peakShape_1_06A.ipf at [http://cires.colorado.edu/jimenez-group/ToFAMSResources/ToFSoftware/Downloads/PIKA/IntermediateRelease/PK_peakShape_1_06A.ipf].
 +
It is also downloadable form the main ToF_AMS sofware download site.
  
* In the peak shape panel, include a string that indicates what species are used but not displayed.(done in 1.04J)
+
==Squirrel==
  
* Make the pop button work for examining one spectra.(done in 1.04J)
+
===To Do Priority===
  
* In HR One Spectra plot, when different open/closed m/z fitting parameters are used, use these parameters in displaying raw data in HR_peakHeights graph (done in 1.04J)
+
* The m/z calibration panel is too slow with the individual ion plots for large data sets.
  
* In HR One Spectra plot, when different open/closed m/z fitting parameters are used, calculate and display sensible 'blue lines, or fits' for the difference spectra in HR_PeakHeights graph (done in 1.04J)
+
* Confirm that the fast mode timing is in time acquiring etc diagnostic waves (Tim O's issue).
  
* Deal with CO tag (Cobalt in usage in tags)  and (C2HN?) (done in 1.04J)
+
* Add the fast ppt link to the wiki
* Change the error that says "no can do!" to something more specific (done in 1.04H at least)
 
* In the tables that indicate pthalate279 use the formula C16H23O4 (done in 1.04H at least)
 
* In Peak shape panel and "live update" button similar to the one in the m/z calibration panel.  Similarly,  don't clear all previous fits when one presses the 'begin peak width' button. (done in 1.04H at least)
 
* In Peak Heights graph, the x axis should be centered on the gray area. (done in 1.04H at least)
 
* In peak shape panel, add the ability to plot real data onto final peak shape graph (show scatter of real points on 'wings') (done in 1.04H at least)
 
* In peak width/shape panel, if todo wave is v mode and user selects a w mode peak shape, an error message should pop up right away. (done in 1.04H at least)
 
* Add the ability to plot residuals on the raw spectra axes in the HR_peakHeights graph (not only on the top axis). (done in 1.04H at least)
 
* In final peak width/shape add ability to subtract sq baseline from peak shape algorithm (instead of baselines being constant)  done in 1.04H at least)
 
* In Samara test case the redo fit button on the peak heights graph wasn't working. (it went to get the raw spectra) (done in 1.04J at least)
 
  
=== Done ===
+
* Change code to automatically set flag for MSSDiff_p to be not in memory if number of runs > x  or in fast mode
  
* Step 4 of pika: maybe rather than two options (one run or Todo wave) and one button, we could just have two buttons?  Something like  “Calculate HR Sticks for averaged Todo wave” and “Calculate HR Sticks for one run”?  (Done in 1.04)
+
* Add the delay at the start/end of run (Tim O's issue) as a diagnostic wave. It is ParVal 88 (89 counting from 1). For now since it doesn't affect the TimeMSOpen and TimeMSClosed values, I will leave it as an optional parameter.
* Change in step 4 options, test for peak height to be less than 0 to be less than user defined value (done in 1.03D)
 
* Warn clearly that changing selected masses is done globally (done in 1.03D)
 
* Add a button in PeakHeights graph to redo pika fits (Keep all user settings from step 4) (done in 1.03D)
 
* Partition Avg ability for todo waves (especially v only, for Qi Chen - done in 1.03A)
 
* In Param graph add radio buttons for raw and percent (done in 1.03D)
 
* In Param graph add print all cursors (done in 1.03D)
 
* Add button for web link in credits panel. (done in 1.03E)
 
* In the graph for one spectra HR fit, move the redo fits button so it is farther away than the arrows. (done in 1.03E)
 
* Double check m/z parameters (so they are not nan) before trying to fit (done in 1.03E)
 
* Alex had an issue with the show inset button being clicked on, but the insets weren't showing up.  Make sure that this toggle value keeps up with the display. (done in 1.03E)
 
  
==Squirrel==
+
* If doing an average mass spectra using the AB correction and the AB correction has nans, the result will be nan - warn the user when this happens (ditto for HR).
  
=== Extreme Priority===
+
* PToF diurnal time trend graphs or daily profiles?
  
* Rework the entire m/z calibration fitting routine so that we fit all plausible peaks (getting peak centers and widths) at the beginning. Then we only need to select which masses to use for the m/z calibration equation.
+
* The average mass spec gives different values in the legend depending on whether you truncate sticks to zero or not. - it should be the same!
  
* Add Doug's threshholding diagnostics and correction factor.
+
* Save PToF DC marker region 1 and 2
  
* Incorporate errors into time series and average mass spectra plots!
+
* Deal with doing Avg PToF for DC marker examine plot
  
 +
* Review Batch table for PAHs
  
===To Do Priority===
+
* Add code to allow normalization of spectra
  
* Tim O wants the capability to write the frag matricies for import into the DAQ.
+
* m/z Cal
 +
** Change the default of the m/z cal ions, to include more ions than just low m/z and W+ (John S)
 +
** ability to fix exponent in m/z cal (John S)
 +
** add warning if exponent is off by more than 0.1% from 0.5 (per John S)
  
* Manjula had the fantastic idea of including a > and < button when looking at individual m/z calibration peaks (could also be used for individual peak width peaks).
+
* In the Baseline panel, average raw spectra section, we need a way to make sure spectra of different ionization types are not averaged together for the display (currently it only distinguishes between v and w).
  
* Change the default resolution values R0 dm, m0 in the baseline panel for W modes?  R0 often gets changed from 500 to ~ 350.
+
* In the Baseline panel, the display shows the stick complement region smoothed twice. (it is fine for the calculation of sticks).
  
* In m/z calibration window, include ability to nan parameters only in runs in a todo wave.
+
* The user has the ability to remove entirely a baseline region between two amus and thus grouping two m/z values into one wide stick (Roya's issue).  A warning should pop up.
  
* In m/z calibration window, for individual peak graphs, add red vertical line for peak center from gaussian fit.
+
* Add Doug's threshholding diagnostics and correction factor.
  
 
* Heringa Maarten brought up the issue about smoothing the AB correction factor when users are doing V/W mode switching. The current code does not smooth across V mode runs when users alternate between v and w mode.  Some code was inserted in the squirrel panel 1.46D, but has not been implemented into the panel.  There needs to be a new field in the AB corrections section of the panel to prevent smoothing across i.e. V-mode runs when adjacent V mode runs are too far apart in time space.   
 
* Heringa Maarten brought up the issue about smoothing the AB correction factor when users are doing V/W mode switching. The current code does not smooth across V mode runs when users alternate between v and w mode.  Some code was inserted in the squirrel panel 1.46D, but has not been implemented into the panel.  There needs to be a new field in the AB corrections section of the panel to prevent smoothing across i.e. V-mode runs when adjacent V mode runs are too far apart in time space.   
  
* The "new only" checkbox near the pre-process button flagged things incorrectly.  (Tim O and Achim's PToF example)
+
* Tim O really want to have PToF min and max size regions for PToF operations.
  
* The save interval section near the get index should perhaps be removed.
+
* Incorporate the CE algorithms as identified by Tim O, Roya, etc
  
* Tim O really want to have PToF min and max size regions for PToF operations.
+
* Implement Mike Cubison's suggestion of the use of pcurser in creating new todo waves.
  
* It needs to be made more clear that the m/z base section in MS tab deals only with raw spectra.
+
* Change the field analysis to warn users to check for large CO influence in ab correction; provide a sample pre and post AB correction from Tim to demonstrate
  
* Leah W found a bug having to do with appending average mass spec data in units of Hz to an existing graph
+
* Add somewhere in the help/faq something about the averaging of raw spectra and the units conversion (Donna - see James email of 11/2007).
  
* Add some sort of pre-compiler directive that helps users that don't have the hdf5 xop loaded.
+
* Doug wants the diagnostic of Closed/Diff sticks.  Anything that sticks out as being very high has a different source (CO2, O, K, C).
  
* Incorporate the CE algorithms as identified by Tim O, Roya, etc
+
* Rework the entire m/z calibration fitting routine so that we fit all plausible peaks (getting peak centers and widths) at the beginning.  Then we only need to select which masses to use for the m/z calibration equation.
  
* Make clear the errors units (They are in Hz, and if a time series and m/z matrix is found (in the MS tab), the units will follow the units of the matrix.
+
* In m/z calibration panel, add vertical red lines for gaussian peak centers. Change black lines to gray.  When saving a copy of the m/z calibration settings, write a message to the history window.
  
* Mike's pcurser in todo wave names
+
* Manjula had the fantastic idea of including a > and < button when looking at individual m/z calibration peaks (could also be used for individual peak width peaks).
  
* Mike Q's issue with m/z calibration with fast mode data
+
* In the m/z calibration panel, think about the ability to tweak settings in conditions of high loadings (Mike C).
  
* In some data sets, many small intermediate files get written instead of a few big ones. (Some of Mike C's Arctas)
+
* Add the ability to use parentheses as grouping symbols in the todo runs control (i.e. "(todo1 or todo1) and todo3" )
  
* In the preprocess step, make clear the distinction between using DAQ sticks and recalcing sticks
+
* A better system for upgrading experiments is needed (stand-alone upgrade ipf).
  
* In average mass spec plots, review how negative values can be 'hidden' by other traces. (Doug thought that it wasn't working right.)
+
* Redo download web page (removing old copies, etc).
  
* In average mass spec plots, a better system for expanding/shrinking is needed.Doug suggests a 'magnification' drop down menu - x 2, x 10, x 25, etc. Also, when checking linear scale have the default go to 0.
+
===To Do (Less Urgent)===
  
* Review existing code that deals with fast mode data.
+
* Add some sort of pre-compiler directive that helps users that don't have the hdf5 xop loaded.
  
* Change the field analysis to warn users to check for large CO influence in ab correction; provide a sample pre and post AB correction from Tim to demonstrate
+
* Tim O wants the capability to write the frag matricies for import into the DAQ.
  
 
* Doug really likes to have time series graphs be such that grids are on midnight of every day, and labels have day of week inserted.
 
* Doug really likes to have time series graphs be such that grids are on midnight of every day, and labels have day of week inserted.
  
* In the m/z calibration panel, think about the ability to tweak settings in conditions of high loadings (Mike C).
+
* In some data sets, many small intermediate files get written instead of a few big ones.  (Some of Mike C's Arctas)
  
* Add somewhere in the help/faq something about the averaging of raw spectra and the units conversion (Donna - see James email of 11/2007).
+
* Add a note to the help web site that tells what the blue/red progress bars mean.
  
* Doug wants the diagnostic of Closed/Diff sticksAnything that sticks out as being very high has a different source (CO2, O, K, C).
+
* Leah noticed that when doing a recursion check when the frag waves are big (5000 m/z)her computer slowed waaay downRevisit this functionality for big frag waves.
  
* In Tim's Ron Brown data set the m/z calibration panel updates so slow (91,000) runs because the right side plots update also.
+
* We may want to help folks who want to combine several squirrel experiments in cases, for example, when users have a 2 month campaign. Create a meta-squirrel?
 
 
* In Tim's Ron Brown data it might be good to write MSSDiff_p matrix to memory to free up memory - make this an option?
 
 
 
* Allow for m/z fitting of PToF data
 
  
* Add a note to the help web site that tells what the blue/red progress bars mean.
+
* Some want the ability to 'shut off' the automatic coloring that happens with a MS time series graph, for example.
  
===To Do (Less Urgent)===
+
* Add the ability to do weekday/weekend diurnal time series plots (Folks can do this right now by making todo waves that reflect weekday/weekend periods.  Still, it would be a convenience.
  
 
* Include Frank D's method for deriving detection limit for CToFs? http://direct.sref.org/amt/2009-2-33
 
* Include Frank D's method for deriving detection limit for CToFs? http://direct.sref.org/amt/2009-2-33
 
* In the MS tab, make more clear the idea that the m/z section only deals with raw spectra
 
  
 
* Deal with the popup menus that deal with long strings by using 'reflection'.
 
* Deal with the popup menus that deal with long strings by using 'reflection'.
  
 
* Doug wants to rename 'AB reference runs' to something else.
 
* Doug wants to rename 'AB reference runs' to something else.
 
* Heikki had out-of-memory issues with the organic matrices.
 
  
 
* Make more consistent the todo wave creation scheme whereby users can input "todowave and not 1xxx and not 2yyy" for the todo wave formula
 
* Make more consistent the todo wave creation scheme whereby users can input "todowave and not 1xxx and not 2yyy" for the todo wave formula
Line 237: Line 253:
  
 
* Incorporate some peak finding code as per Jesse's request.
 
* Incorporate some peak finding code as per Jesse's request.
 
* Deal with Fast runs not finishing a fast mode cycle.
 
 
* Update squirrel web site.
 
  
 
* From Ken: In the baseline fitting panel, when scrolling through the baseline fits for all runs in a todo list, add the ability to pause on a chosen run.  It would also be great to select the direction in which the run baselines are scrolled - forward or backward.
 
* From Ken: In the baseline fitting panel, when scrolling through the baseline fits for all runs in a todo list, add the ability to pause on a chosen run.  It would also be great to select the direction in which the run baselines are scrolled - forward or backward.
  
* Create Frag table default fragment text waves (Donna would be good for the manual)
+
* Create Frag table default fragment text waves (Donna - this would would be good for the manual)
  
* Tim O's issue with somehow getting the squirrel_index matrix messed up and getting the stick matrix data set being as wide (columns) as the raw spectra matricies. (Donna - is this replicable?)
+
* Tim O's issue with somehow getting the squirrel_index matrix messed up and getting the stick matrix data set being as wide (columns) as the raw spectra matrices. (Donna - is this replicable?)
  
 
* Review the use of t_series_old
 
* Review the use of t_series_old
Line 255: Line 267:
  
 
* Add an upper limit to when we generate normalization factors (current usage is to have user's create custom size interval).
 
* Add an upper limit to when we generate normalization factors (current usage is to have user's create custom size interval).
 
* Amewu gets (1) strange wave stats error in baseline panel (2) strange interpolate2 error after pre-processing
 
  
 
* Suggestion from Tim O: Since everyone needs to do size calibrations, can we get the fitting function into the squirrel code for use?  Ideally, we would have two functions, one to fit velocity vs dva data (typical process for getting size calibration parameters) and one that takes the parameters and calculates the dva from velocity (i.e. same equation, just solved for dva rather than velocity).  I can provide you with both, if you wish.  I keep finding myself going back to either retype these in all of the time or copying for older *.pxp files.
 
* Suggestion from Tim O: Since everyone needs to do size calibrations, can we get the fitting function into the squirrel code for use?  Ideally, we would have two functions, one to fit velocity vs dva data (typical process for getting size calibration parameters) and one that takes the parameters and calculates the dva from velocity (i.e. same equation, just solved for dva rather than velocity).  I can provide you with both, if you wish.  I keep finding myself going back to either retype these in all of the time or copying for older *.pxp files.
Line 262: Line 272:
 
* Achim wants the ability to calculate sticks beyond 1000m/z for SP2. (It is a simple code change; Achim knows how to do it.  It is not clear how this could be incorporated into squirrel for other users.)
 
* Achim wants the ability to calculate sticks beyond 1000m/z for SP2. (It is a simple code change; Achim knows how to do it.  It is not clear how this could be incorporated into squirrel for other users.)
  
* Add a button to the misc tab for dumping MSSDiff_p to intermediate files (Tim O Ron Brown data).
+
===To Discuss===
  
 
+
* Leah mentioned that in the legend for the average mass spectra plots the speciated total values do not use the RIE for each species.  They are just the nitrate equivalent sum. Should we change the total values in the legend to reflect the species RIE values?  Perhaps just make it more obvious in the legend? Similarly, when one truncates the sticks to zero, should the values in the legend be the sum of the truncated or the pre-truncated values?
===Done in latest beta 1.46X===
 
 
 
* Fixed a bug found by Paul Reitz that had to do with finding an airbeam correction using non-autoset and not multiple tof types or ionization schemes. (Fixed in SQ_AMSPanel_1_46A.ipf, download [http://cires.colorado.edu/jimenez-group/ToFAMSResources/ToFSoftware/Downloads/Squirrel/IntermediateRelease/SQ_AMSPanel_1_46A.ipf SQ_AMSPanel_1_46A.ipf])
 
 
 
* Fixed a bug that had to do with displaying raw mass spectra with the squirrel baselines removed (Fixed in SQ_MzCalBkgnd_1_46.ipf, download [http://cires.colorado.edu/jimenez-group/ToFAMSResources/ToFSoftware/Downloads/Squirrel/IntermediateRelease/SQ_MzCalBkgd_1_46A.ipf SQ_MzCalBkgd_1_46A.ipf])
 
 
 
 
 
===To Discuss===
 
  
 
* Normalize w mode to v mode - One diagnostic:the ratio of v and w sticks with the x axis as m/z.
 
* Normalize w mode to v mode - One diagnostic:the ratio of v and w sticks with the x axis as m/z.
Line 281: Line 283:
 
** Frag_SO3 and H2SO4 are kept in Squirrel for reference and if someone wants to put them back to use, but not used by default
 
** Frag_SO3 and H2SO4 are kept in Squirrel for reference and if someone wants to put them back to use, but not used by default
  
* change frag_organics as per Allison's paper
+
* Change frag_organics as per Allison's paper
  
 
* Should we allow users to enter a negative value for the PToF vl parameter? (Liz, John J)
 
* Should we allow users to enter a negative value for the PToF vl parameter? (Liz, John J)
  
 +
* Should I add PAH to the default list of species? (i.e. in the MS average graphs)
  
===Done in version 1.45===
+
===Completed===
  
* Puneet discovered a small error in the m/z calibration panel.  If he adds a new species but doesn't use it to fit the section of code that deals with the updating of individual graphs throws an error. (fixed in 1.44B)
+
* Incorporate John S's method for improving m/z calibration using PFT (or PFK) (via a constant power value) 1.49E.
  
* Bug found with Sanna: when using a user defined time base but minutes = 0, we get an out of memory error. (fixed in 1.44B)
+
* Tim O made a good suggestion that we optionally kill some waves in the ms_mats and HR_ms_mats folder when we are done with them.  This interface will likely look like a checkbox in the misc tab. (This is a button in the misc tab.) 1.49E
  
* Write intermediate files as compressed HDF files?  Check to see if/how this is best done for updating experiments. (Jose's suggestion only NEW files are compressed) (done in 1.44B - only new files are gzip compressed)
+
* In m/z calibration window, include ability to nan parameters only in runs in a todo wave.
  
* Tim O et al made the suggestion for using the daq values as input for the m/z calibration fit (done in 1.44B - this seems to make the m/z calibration routine 10 - 20% faster)
+
* In the m/z calibration panel we should use the term 'resolving power' instead of 'resolution'.
  
* Error that Samara found when trying to get error matricies... The dimensions of the stick matrices were different for open and closed (done in .144B)
+
* Allow for m/z fitting of PToF data
 
 
* In mz_reSizeMzCalWaves() recode it so that it does not nan the Accuracy_XX_yy waves when one fits only one run. (fixed in 1.44B)
 
 
 
* There are still some instances where if the user presses a button when not in the root folder, squirrel is ungraceful (Niall). (fixed many in 1.44B)
 
 
 
* When doing an m/z cal fitting, put runs with bad fits into a separate todo wave. (done in 1.44B)
 
 
 
* Deal with Helsinki/Doug's unique PToF data (grouping last 3 of 4 choppers).  Done but only distributed for use in Mikael's data
 
 
 
* Incorporate speccorr_list (Batch table) functionality to make species specific time series corrections via batch table. (Done in earlier versions.  Ken entered this item but was not clear on its usage)
 
 
 
* The 'resolution' parameters for determining peak integration area need to be printed to the history window whenever we recalc sticks. (Done in 1.44B)
 
  
* Pre-process: when recalculating sticks with new m/z parameters, the error related to some m/z parameters being NaNs often shows up.  A few times at the beginning, and then halfway through.  Is there any way to deal with this more effectively? (done in 1.44B - only tested for fast mode situations but other situations should equally apply)
+
* Paul W made the suggestion of naming the intermediate files using all or part of the same name as the experiment (instead of the AMS_intermediate_xx.hdf naming scheme).
  
* When overwriting a todo wave the message that gets printed to history can be incorrect - it says that the current todo wave was overwritten. (Done in 1.44B)
+
* Redo the Frag Corrections section to have just one big panel with pop-able graphs.
  
* From James: If you get it to do a batch of diurnal plots, it calculates all of the time series afresh before plotting each graph. It's just a function call on the wrong side of a for statement, so it should be pretty trivial to fix. It also plots a graph of the time series each time around as well, which is really annoying because it ends up creating loads of excess graphs. (Done in 1.44B)
+
* In the baseline panel the setting the R0 etc values to default button does not work. (Fixed in 1.49E)
  
* In the baseline subpanel, make the mass defect default values more clear. (Done in 1.44B)
+
* The names for the average raw mass spec should be changed to not have the "_all" suffix. (done in 1.47I)
  
* In the modify SI table, also include ionization and tof type waves so users can more easily see how to change SI values. (Done in 1.44B)
+
* Add a button to the misc tab for dumping MSSDiff_p to intermediate files (Tim O Ron Brown data). (done in 1.47I)
  
* Add the LS diagnostics to the set of waves that get loaded. (finished in 1.43I, checked in 1.44B)
+
* Users may often run into out of memory issues when performing the Export Matricies in the PToF tab, because results will be #runs * #mz * #sizebins big. Perhaps there should be an option for exporting these matricies, but cutting things off a a max m/z value (150ish?) (done in 1.47I)
  
* Load in menu information from DAQ hdfs and make menu info more readily available to user. (kinda finished in 1.44B) - we have loaded in the tofType and ionizationType since ~ 5 versions ago. This is all that most will ever need. I have added DAQsamplingType for users like Doug who may have set up funky menus.  We have chosen to avoid having menu numbers as indicators/mask waves because they really don't provide the details we need - the toftype and ionizationtype.
+
* In the preprocess step, make clear the distinction between using DAQ sticks and recalcing sticks.(fixed in 1.47I)
  
* Make squirrel handle situations whereby the sEI is not really sEI but a different menu. (done via use of samplingType - see note above, done in 1.44B)
+
* Appending one size distribution on top of another (with a different todo wave) does not work.(fixed in 1.47I)
  
* Add 2 new ionization types to code (see Joel's email of March 2008). (done in 1.44B)
+
* In the modify SI table, perhaps include the TofTypeVKey and TofTYpeWKey waves to help folks know that they are available for use.(done in 1.47I)
  
* In m/z calibration panel we need a readout that tells ionization type (sEI, EI) as well as ToF type (v/w) (done in 1.44B)
+
* In baseline panel, when averaging many spectra, if run num > 1e6, temporary waves have an E+ in them (Liz Rainer data) (done in 1.47I).
  
* In baseline panel we need a readout that tells ionization type (sEI, EI) as well as ToF type (v/w)(done in 1.44B)
+
* In the MS tab, when one chooses to update an average mass spec plot, the legend should update (done in 1.47I).
  
* Add code that somehow simply tells the user whether the sticks were recalced or not, and if recalced, the parameters used. (done in 1.44B)
+
* In the MS tab, perhaps move all the average mass spec controls to the left, and the buttons to the right so that users don't hit one when they mean to hit the other.
  
* When a user enters runs and the first is > than the last, 45-40, a better error message should be generated. (Now it generates a todowave with 0 points, done in 1.44B)
+
* Tim O's issue with HDF file generating errors for not being able to append to hdf files. The code needs to add the checking of errors generated during the routine closing of hdf files (done in 1.47I).
  
* From Ken: In the baseline fitting panel make the main plot in the popped window automatically scale the x axis to that in the main window.  (done in pre-144B? If the user uses the marquee instead of the panel settings for setting the axis range, the x,y scales won't be the same. But if they use the panel variables for x and y scales, it works.)
+
*In pre-processing section, change the Diff Sticks = Open Sticks - Closed Sticks to that most users don't use it (Leah W) (done in 1.47I)
  
* In Dva popped table we should also see run numbers and whether or not a run has PToF data. (done in 1.44B)
+
* The list of completed programming tasks has been moved to [http://cires.colorado.edu/jimenez-group/wiki/index.php/Sq_Pk_Programming_Completed#Squirrel http://cires.colorado.edu/jimenez-group/wiki/index.php/Sq_Pk_Programming_Completed#Squirrel]
 
 
* Ask user to save experiment before blacklisting (done in 1.44B)
 
 
 
* Added the possible creation of an allMS todo wave so that users can be clear when there is no MS data (only PToF).
 
 
 
* James would like the ability to enlarge the stick integration region for one m/z.  Currently the code only allows you to make the stick integration region smaller. (done in 1.44B)
 
 
 
* In the baseline panel allow uses to go higher than m/z 500.(done in 1.44B)
 
 
 
* We need to be able to select some ions to be used in m/z cal and peak shape for V or W only (this was always available and now users can save m/z data sets in 1.44B).
 
 
 
* Tim O made the suggestion that there be a little button for saving the m/z calibration table with a todo suffix. (done in 1.44B)
 
 
 
* Allow users to go > 500m/z in baseline panel. (done in 1.44B)
 
 
 
* Review the use of custom time series - are sanity checks in place? (checks added in 1.44B)
 
 
 
* In frag (Alice's) diagnostic plots, change legends that give integer m/z values to molecular species.
 
 
 
* We have problems with averaging PToF runs when the PToF settings change between runs.  Have squirrel automatically create different todo waves for each PToF setting. (done in 1.44D)
 
 
 
* When a user has applied an AB correction and a new ion_eff has been calculated, then adds new hdf files (and indexes, etc) the ion_eff wave may not grow to the needed size.  This adding more hdf files scenario could also cause problems when new ionization types (sEI) may appear.  (Donna: email from Alberto Presto July 2008 - he used sEI 'mode' when it wasn't soft EI, he just wanted to clearly delineate modes, and we now have sample types to address this).  (done mostly in 1.44D - more warnings pop up when the number of points in corr_fact wave is not the same as t_series)
 
 
 
* sq_verifyTodoForPToF function aborts in case where every run in ToDo wave has PTOF. Currently commentize the abort call to get SQ to pre-process. MJC 1/14/09 (done, I believe, in 1.44A)
 
 
 
* Manjula thought that when the user puts more hdf files in the data folder, and presses the Get Index button, that the code does not handle new v/w switching and mode changes well. (For example, at first the DAQ is set up to do all v then does v/w switching.)  Donna's comment: Right now I cannot reproduce this problem but there was a problem with the diagnostic plot not updating correctly, and this was fixed in 1.43F. (refixed in 1.44D)
 
 
 
* In frag check tab we need a button to update plots instead of replotting every time. (fixed in 1.44D)
 
 
 
* In frag diagnostic plots we need to change the 'display gaps' setting.(fixed in 1.44D)
 
 
 
 
 
===Done in version 1.44===
 
 
 
* Check the labels for graphs generated using x vs tof and dxdlogdp (Done in 1.43I).
 
* Possibly rename the checks to be done before preprocessing as 'stick adjustments'. (Done in 1.43I)
 
* Change defaults in Frag checks tab to be 'all' and colored by timewave.
 
* Fix bug that pops up in Frag checks plots. (Done in 1.43I)
 
* Make sure that when doing DC marker corrections squirrel changes todo wave to be todo wave AND allPToF (Donna note:  This was done primarily to avoid creation of blanks in the prepocess step but was also added in the corrections tab, done in 1.43I).
 
* Add checkbox in Misc - graphs section that sets the default displays to not show gaps. (Done in 1.43I)
 
* When preprocessing PToF sticks generate a todo wave that has only PToF runs in it instead of generating dire warning. (Done in 1.43I)
 
* Change the preprocess checkbox settings so that the default is to apply dc markers. (Done in 1.43I)
 
* An error pops up when trying to creat normalization factors for user defined species (As in Pb for Leah). (The normalization routine was overhauled in 1.43G and this error should no longer appear.)
 
* A general warning should pop up when users try to create todo wave names with > x characters. (Done in 1.43E)
 
* Add button to generate diagnostic plot if user killed it. (Done in 1.43I)
 
* Change default for average mass spec so that only sticks (not sticks + raw) are calculated. (Done in 1.43I)
 
* Bug found by Mike Q about checking modifications of all frag waves, not just species selected in average mass spec (fixed in 1.43C-ish)
 
* In the m/z calibration panel, make the live option checked automatically when viewing one run. (Done in 1.43I)
 
* Bug found by Carly whereby the MS time series for the species all can replace the 'all' todo wave (Done in 1.32I)
 
* For PToF size distributions, the total loading for each species should be displayed (somewhat similar to the legends that get displayed with average mass spec). (Total MS and normalization factors are now printed to history - Done in 1.43I)
 
* When an intermediate file was not accessed successfully change the code from the existing error message that gets printed to history, to an abort command (Manjula).  Done in 1.43I
 
  
 
==Time Trace==
 
==Time Trace==
Line 400: Line 340:
 
==LS==
 
==LS==
  
===Priority===  
+
=== To Do ===  
  
 +
Get the firstBeta release out.
  
 
=== To Do ===
 
 
Wait for requests and updates from Eben, Tim, Jay, etc.
 
  
 
==EC==
 
==EC==
  
===Priority===  
+
===To Do===  
 
* Wait for request from Delphine - nothing to do yet.
 
* Wait for request from Delphine - nothing to do yet.

Latest revision as of 16:19, 19 May 2011

APES

To Do

  • Add the ability to do EA on non HROrg species.
  • Add more web documentation, with specific examples of mass calculations.

Pika

To Do Priority

  • Do some sort of smoothing for the peak shape when user uses different left/right parameters.
  • The HR ion mass calculator, Query tool seems to need some sort of initialization before it works.
  • Remove use of HR_specName_list because it is redundant
  • for PMF prep error panel:
    • Jose wants the running median in the time series of the error and noise times series for individual sticks.
    • Do the time series plots differently so that users don't have to change axes
    • Graphs for isotopes hide isotopes
    • In prepping for PMF, nan isotopes
    • Add spike removal similar to what Qi Zhang did?
  • Ask J if we have volunteer for the keeping of the HR list
  • Generate NeighboringHRIonSignal new type of Pika error
  • Pika fitting graph - make it so that the user can resize
  • Doug wants H2O, OH, O and CO to be colored as CO2 in mass spec Org family plots.
  • Create an HR frag check panel
  • Add ability to plot in time series closed baseline sticks.
  • Check that m/z calibration panel works if one run isn't done first.
  • Add K, j41K to default list of ions to fit
  • For the time-varying peak width, have a better errror if the user doesn't select an HR ion that was used.
  • Remove N3 from default list. (Done?)
  • Add code to require and check that HR ions be formatted in a certain way - i.e. disallow H7C5 and require C5H7.
  • Change code so that for todo waves fit with different pw and ps but with the same set of HR ions, one can nicely generate time series or average mass spectra. The code should check that the set of HR ions are the same for different todo waves. If they are the same, then one should be able to proceed.
  • Add code to check that all the HR ions within each frag table element is fit. (Note: this is hard to do because of the recursive nature of frag entries!)
  • Make a Pika-specific airbeam correction?
  • Implement the "resolution" fitting function for the peak width panel as default (Pete's suggestion)
  • Allow including other families (such as SO, NO) in elemental analysis along with HROrg (Puneet & Doug)
  • Tim O made a good suggestion for instances where peaks are not fit (resulting in nans). Perhaps one could compare the squirrel signal at an integer m/z, and if the value is sufficiently small, (the raw spectra shows noise), all the peak height values could be set to 0.
  • We may want to allow a 'smoothly varying' peak width.
  • Change all peak shape candidates to have a 0 entry in the left- and right-most value so that 'peak-shape type baselines' dont' get extended beyond +- 7 PW values.
  • Fully fold in Mike Cubison's sensitivity analysis code.
  • Many parameters panel
    • add small raw spectra insets to left and right
    • add graph showing spectra-like data with window highlighting where you are
    • have run numbers update with cursor move
  • Add ability to duplicate PeakHeights graph and functionality
  • In the m/z calibration panel, currently the width and accuracy vs m/z graphs don't show a summary of all the runs. E.g. the summary of all the peak widths for all the runs with error bars in the peak shape panel very nicely summarizes this information.
    • (Jose) This could be done with a radio button on top of the graph that selects SR/AR (meaning single run / all runs)

To Do (Less Urgent)

  • In HR_peakHeights_gr, change the color of the residuals to be different from the red, orange, brown. Also add squirrel integration regions. Also, add a checkbox to add remove black lines between raw data points.
  • Perhaps shorten the names of some of the new error estimates (Pete)
  • For Pika, we need to incorporate some sort of algorithm that sets the minimum error to be 1 ion.
  • In average mass spec plots (including the 5-graph panel and the elemental analysis) Doug wants a way to 'move' isotopic child HR ions to their parent ion (i.e. j13C would be moved and grouped with the HR ion C.)
  • From the HR_peakHeights_gr, when adding a brand new mass to the table, add a note in the history. Also add a message when adding and removing masses to fit?
  • m/z calibration and masses - We need to subtract the mass of an electron from all our masses. Note from Donna: I want to leave this decision up to the HR list committee (Puneet, Niall, Delphine)
  • Generate 'Neil's graph' whereby we plot as sticks from zero amount of C and H mass as a function of number of Oxygens. Do a similar thing for the number of C's (the amount of H and Os and a function of number of Cs).
  • In PeakWidth/Shape panel, add ability to not do average PW fits using the std dev as weights
  • In the Graph 5 summaries, include one 'bar' type graph showing the fraction of each family to the total.
  • Pika on PToF spectra (gasp!)
  • There is a need to consolidate the three sets of m/z lists that exist into a single one. We currently have the list of m/z possibly used in fits,(1) ones actually selected for HR sticks (2) list of m/z used for m/z calibration, and (3) the list of isolated ions. Each list is a subset of the previous one, and we can work with a single big list, and a series of masks to select / deselect the various m/z for the different purposes.
  • If user only does the m/z calibration and does not play with the HR panel, the code calculating the HR sticks may go awry (maybe - I need to check).
  • Think about NH4 measure vs predicted and RIE for NH4 (Donna: Jose's July 2008 email).
  • Make clear what limitations there are about the total number of masses that can be fit at any m/z value.

To Discuss

  • Errors for input to PMF.

Done

Done in 2010 Users Mtg release:

  • Doug wants a more clear warning when masses not fit (most likely due to nonmonotonicity in peak shape) in one spectra or many.
  • In Diurnal plots, x axis label is on y axis.
  • Remove any mention of "fitting with SQ baseline subtracted in the panel", and replace with "fitting on top of the SQ baseline" (Jose)
  • We need to add camels, dromedaries, and vw "bug" ions to the lists. We need some notation for these ions, let me suggest vw28, vw23, d28, d32, and c28, c32, with the comment column describing what they are. It is important that those are lowercase letters, so people don't confuse these ions with vanadium (V), tungsten (W), and deuterium (D).
  • Jose wants to add other types of errors for folks to plot/use. If the current list of errors is considered to be a,b,c,d, we also want sqrt(a^2+d^2) and sqrt(a^2+b^2+d^2).
  • In HR_PeakHeights graph perhaps color the individual peak shapes to the colors of the families.
  • Begin to implement the HR frag table. W need to introduce the concept of species. For example CO2 would belong partially to an HR-Organic and partially to the HR_air species. This also involves the use of an HR batch table, In this HR batch table users also their own RIE, CE. Note that this is a similar but slightly different concept than the family groupings.
  • Change default behavior so that we have two HR difference sticks one which is found by fitting the raw diff spectra (called HRDiff), and an OpenMinusClosed which is simply the open HR sticks - closed HR sticks.
    • Change the default behavior from fitting difference spectra to only fitting open and closed. James' point at the user's mtg is that we should always be looking at the HR_OpenSticks - HR_ClosedSticks.
    • Jose, 4-Apr-09: disagree with not fitting the diff, I think we should keep fitting it as for some peaks it is much more useful. However we should also show the open-closed fits (not just sticks) in the PIKA window. User should be able disable fitting the diff, and to use the diff results, even if the default it to use the open-closed results.
  • The peak width/shape panel needs tweaks to be more user friendly. The specific changes have not yet been worked out.

Important Bugs fixed since last release

There was an annoying bug that resulted in not being able to choose > 1 mass fragment each for the left and right-side peak shape candidate. The code was fine in 1.05 and using Igor version < 6.10. But in Igor 6.10 there was a compilation error, and the quick fix for Igor 6.10 caused this behavior. You can download PK_peakShape_1_06A.ipf at [1]. It is also downloadable form the main ToF_AMS sofware download site.

Squirrel

To Do Priority

  • The m/z calibration panel is too slow with the individual ion plots for large data sets.
  • Confirm that the fast mode timing is in time acquiring etc diagnostic waves (Tim O's issue).
  • Add the fast ppt link to the wiki
  • Change code to automatically set flag for MSSDiff_p to be not in memory if number of runs > x or in fast mode
  • Add the delay at the start/end of run (Tim O's issue) as a diagnostic wave. It is ParVal 88 (89 counting from 1). For now since it doesn't affect the TimeMSOpen and TimeMSClosed values, I will leave it as an optional parameter.
  • If doing an average mass spectra using the AB correction and the AB correction has nans, the result will be nan - warn the user when this happens (ditto for HR).
  • PToF diurnal time trend graphs or daily profiles?
  • The average mass spec gives different values in the legend depending on whether you truncate sticks to zero or not. - it should be the same!
  • Save PToF DC marker region 1 and 2
  • Deal with doing Avg PToF for DC marker examine plot
  • Review Batch table for PAHs
  • Add code to allow normalization of spectra
  • m/z Cal
    • Change the default of the m/z cal ions, to include more ions than just low m/z and W+ (John S)
    • ability to fix exponent in m/z cal (John S)
    • add warning if exponent is off by more than 0.1% from 0.5 (per John S)
  • In the Baseline panel, average raw spectra section, we need a way to make sure spectra of different ionization types are not averaged together for the display (currently it only distinguishes between v and w).
  • In the Baseline panel, the display shows the stick complement region smoothed twice. (it is fine for the calculation of sticks).
  • The user has the ability to remove entirely a baseline region between two amus and thus grouping two m/z values into one wide stick (Roya's issue). A warning should pop up.
  • Add Doug's threshholding diagnostics and correction factor.
  • Heringa Maarten brought up the issue about smoothing the AB correction factor when users are doing V/W mode switching. The current code does not smooth across V mode runs when users alternate between v and w mode. Some code was inserted in the squirrel panel 1.46D, but has not been implemented into the panel. There needs to be a new field in the AB corrections section of the panel to prevent smoothing across i.e. V-mode runs when adjacent V mode runs are too far apart in time space.
  • Tim O really want to have PToF min and max size regions for PToF operations.
  • Incorporate the CE algorithms as identified by Tim O, Roya, etc
  • Implement Mike Cubison's suggestion of the use of pcurser in creating new todo waves.
  • Change the field analysis to warn users to check for large CO influence in ab correction; provide a sample pre and post AB correction from Tim to demonstrate
  • Add somewhere in the help/faq something about the averaging of raw spectra and the units conversion (Donna - see James email of 11/2007).
  • Doug wants the diagnostic of Closed/Diff sticks. Anything that sticks out as being very high has a different source (CO2, O, K, C).
  • Rework the entire m/z calibration fitting routine so that we fit all plausible peaks (getting peak centers and widths) at the beginning. Then we only need to select which masses to use for the m/z calibration equation.
  • In m/z calibration panel, add vertical red lines for gaussian peak centers. Change black lines to gray. When saving a copy of the m/z calibration settings, write a message to the history window.
  • Manjula had the fantastic idea of including a > and < button when looking at individual m/z calibration peaks (could also be used for individual peak width peaks).
  • In the m/z calibration panel, think about the ability to tweak settings in conditions of high loadings (Mike C).
  • Add the ability to use parentheses as grouping symbols in the todo runs control (i.e. "(todo1 or todo1) and todo3" )
  • A better system for upgrading experiments is needed (stand-alone upgrade ipf).
  • Redo download web page (removing old copies, etc).

To Do (Less Urgent)

  • Add some sort of pre-compiler directive that helps users that don't have the hdf5 xop loaded.
  • Tim O wants the capability to write the frag matricies for import into the DAQ.
  • Doug really likes to have time series graphs be such that grids are on midnight of every day, and labels have day of week inserted.
  • In some data sets, many small intermediate files get written instead of a few big ones. (Some of Mike C's Arctas)
  • Add a note to the help web site that tells what the blue/red progress bars mean.
  • Leah noticed that when doing a recursion check when the frag waves are big (5000 m/z)her computer slowed waaay down. Revisit this functionality for big frag waves.
  • We may want to help folks who want to combine several squirrel experiments in cases, for example, when users have a 2 month campaign. Create a meta-squirrel?
  • Some want the ability to 'shut off' the automatic coloring that happens with a MS time series graph, for example.
  • Add the ability to do weekday/weekend diurnal time series plots (Folks can do this right now by making todo waves that reflect weekday/weekend periods. Still, it would be a convenience.
  • Deal with the popup menus that deal with long strings by using 'reflection'.
  • Doug wants to rename 'AB reference runs' to something else.
  • Make more consistent the todo wave creation scheme whereby users can input "todowave and not 1xxx and not 2yyy" for the todo wave formula
  • Change PToF code so that users have an option to plot the legend (for size distribution graphs).
  • For PToF size distributions, often the size range extends to smaller values than are useful. Add controls so that user can quickly zoom in and out of interested size ranges.
  • Add the ability to calculate diameter mean and median in PToF data (Manjula).
  • Occasionally after preprocessing, the status bar ends saying “Estimating space requirements” rather than “Done” (Donna can't reproduce this)
  • For DC marker corrections, Manjula wants to be able to enter nitrate, and then have squirrel figure out what m/zs this corresponds to.
  • Re-look at IE/AB calculations (Donna - see Tim O's email of Jan 2008).
  • In the diagnostic plots, perhaps the PToF airbeam wave should be nanned whenever the run doesn't have PToF data.
  • In m/z calibration panel, check the possible bug when interpolating across a todo wave.
  • Review todo wave name length limitations.
  • In the m/z calibration panel, make more clear what the nan param button does.
  • From Tim O:During manual (F3) saves, the mass spectral data (e.g. MSOpen_V, MSSOpen_V, etc.) are saved as simple data arrays with the m/z in the rows direction. During autosaving, even if only for one run, these waves are always saved with the m/z in the column (run number in the row dimesion). It turns out that Squirrel can read and process either, just not at the same time. Thus, while he can process the data separately, he cannot ask Squirrel to load and process autosaved data with nonautosaved data interspersed.
  • Incorporate some peak finding code as per Jesse's request.
  • From Ken: In the baseline fitting panel, when scrolling through the baseline fits for all runs in a todo list, add the ability to pause on a chosen run. It would also be great to select the direction in which the run baselines are scrolled - forward or backward.
  • Create Frag table default fragment text waves (Donna - this would would be good for the manual)
  • Tim O's issue with somehow getting the squirrel_index matrix messed up and getting the stick matrix data set being as wide (columns) as the raw spectra matrices. (Donna - is this replicable?)
  • Review the use of t_series_old
  • Review the use of the todo graph.... add checkboxes to remove todo waves?
  • Add code that calculates the mean/median for size distributions (Manjula)
  • Add an upper limit to when we generate normalization factors (current usage is to have user's create custom size interval).
  • Suggestion from Tim O: Since everyone needs to do size calibrations, can we get the fitting function into the squirrel code for use? Ideally, we would have two functions, one to fit velocity vs dva data (typical process for getting size calibration parameters) and one that takes the parameters and calculates the dva from velocity (i.e. same equation, just solved for dva rather than velocity). I can provide you with both, if you wish. I keep finding myself going back to either retype these in all of the time or copying for older *.pxp files.
  • Achim wants the ability to calculate sticks beyond 1000m/z for SP2. (It is a simple code change; Achim knows how to do it. It is not clear how this could be incorporated into squirrel for other users.)

To Discuss

  • Leah mentioned that in the legend for the average mass spectra plots the speciated total values do not use the RIE for each species. They are just the nitrate equivalent sum. Should we change the total values in the legend to reflect the species RIE values? Perhaps just make it more obvious in the legend? Similarly, when one truncates the sticks to zero, should the values in the legend be the sum of the truncated or the pre-truncated values?
  • Normalize w mode to v mode - One diagnostic:the ratio of v and w sticks with the x axis as m/z.
  • Change frag table of sulfate according to suggestion from Ann
    • Current frag_sulphate -> frag_sulphate_old
    • New frag_sulphate doesn't depend on frag_SO3 and frag_H2SO4 (Ann will wori up this new table)
    • Frag_SO3 and H2SO4 are kept in Squirrel for reference and if someone wants to put them back to use, but not used by default
  • Change frag_organics as per Allison's paper
  • Should we allow users to enter a negative value for the PToF vl parameter? (Liz, John J)
  • Should I add PAH to the default list of species? (i.e. in the MS average graphs)

Completed

  • Incorporate John S's method for improving m/z calibration using PFT (or PFK) (via a constant power value) 1.49E.
  • Tim O made a good suggestion that we optionally kill some waves in the ms_mats and HR_ms_mats folder when we are done with them. This interface will likely look like a checkbox in the misc tab. (This is a button in the misc tab.) 1.49E
  • In m/z calibration window, include ability to nan parameters only in runs in a todo wave.
  • In the m/z calibration panel we should use the term 'resolving power' instead of 'resolution'.
  • Allow for m/z fitting of PToF data
  • Paul W made the suggestion of naming the intermediate files using all or part of the same name as the experiment (instead of the AMS_intermediate_xx.hdf naming scheme).
  • Redo the Frag Corrections section to have just one big panel with pop-able graphs.
  • In the baseline panel the setting the R0 etc values to default button does not work. (Fixed in 1.49E)
  • The names for the average raw mass spec should be changed to not have the "_all" suffix. (done in 1.47I)
  • Add a button to the misc tab for dumping MSSDiff_p to intermediate files (Tim O Ron Brown data). (done in 1.47I)
  • Users may often run into out of memory issues when performing the Export Matricies in the PToF tab, because results will be #runs * #mz * #sizebins big. Perhaps there should be an option for exporting these matricies, but cutting things off a a max m/z value (150ish?) (done in 1.47I)
  • In the preprocess step, make clear the distinction between using DAQ sticks and recalcing sticks.(fixed in 1.47I)
  • Appending one size distribution on top of another (with a different todo wave) does not work.(fixed in 1.47I)
  • In the modify SI table, perhaps include the TofTypeVKey and TofTYpeWKey waves to help folks know that they are available for use.(done in 1.47I)
  • In baseline panel, when averaging many spectra, if run num > 1e6, temporary waves have an E+ in them (Liz Rainer data) (done in 1.47I).
  • In the MS tab, when one chooses to update an average mass spec plot, the legend should update (done in 1.47I).
  • In the MS tab, perhaps move all the average mass spec controls to the left, and the buttons to the right so that users don't hit one when they mean to hit the other.
  • Tim O's issue with HDF file generating errors for not being able to append to hdf files. The code needs to add the checking of errors generated during the routine closing of hdf files (done in 1.47I).
  • In pre-processing section, change the Diff Sticks = Open Sticks - Closed Sticks to that most users don't use it (Leah W) (done in 1.47I)

Time Trace

Not Urgent

  • Make the quick view program instructions more clear.


LS

To Do

Get the firstBeta release out.


EC

To Do

  • Wait for request from Delphine - nothing to do yet.