Difference between revisions of "PMF-AMS Analysis Guide"

From Jimenez Group Wiki
Jump to: navigation, search
(In PIKA)
(Creating the Organics and Error Matrices)
Line 151: Line 151:
 
:If the output does not contain these lines, go back to Step 2 of this section to examine errors that might be reported by PMF in the file PMF2.LOG.
 
:If the output does not contain these lines, go back to Step 2 of this section to examine errors that might be reported by PMF in the file PMF2.LOG.
  
== Creating the Organics and Error Matrices ==
+
== Creating the Organics and Error Matrices (Step 0) ==
  
 
=== In the Q-AMS Software (James') ===
 
=== In the Q-AMS Software (James') ===
Line 160: Line 160:
  
 
=== In PIKA ===
 
=== In PIKA ===
 +
 +
=== Deleting NaNs/zeros ===
 +
 +
=== Recommended Practice: Setting a Minimum Error ===
 +
 +
=== Recommended Practice: Downweight "Weak" Variables ===
  
 
== Perform PMF Analysis *Step 1* ==
 
== Perform PMF Analysis *Step 1* ==

Revision as of 17:01, 22 August 2008

Introduction

The PMF Evaluation Panel consists of 3 Igor procedure files (ipfs) called PMF_Execution, PMF_ViewResults, and PMF_Scatter. This wiki serves as the help and documentation for the software. To run PMF with the panel, the PMF executable and associated files, accessed separately, are required (see Section 3, Installing PMF with Igor). The PMF executable is compiled only for Windows/DOS. It is possible to execute PMF on a Windows computer and then analyze the experiment on a Macintosh.

The ipfs were written by Ingrid Ulbrich and Donna Sueper (Jimenez Group, University of Colorado, Boulder) and Greg Brinkman (Hannigan Group, University of Colorado, Boulder). Questions about this codecan be addressed to Ingrid or Donna at ulbrich@colorado.edu or dsueper@colorado.edu.

PMF (Positive Matrix Factorization) was developed by Dr. P. Paatero (Dept. of Physics, University of Helsinki). One of the original papers describing this method is Paatero, 1997 P. Paatero, Least squares formulation of robust non-negative factor analysis, Chemometrics and Intelligent Laboratory Systems 37 (1997), pp. 23–35. First time users to PMF are encouraged to first read the documentation by Paatero regarding PMF (see Section 9, Other Resources).

This Igor toolkit was intended for use in analyzing AMS data, but there are only few assumptions in the toolkit relating to AMS-type data. Non-AMS users of this software can skip Section 4, Creating the Organics and Error Matrices.

A Message to Contributors

We want to encourage active participation by all users in the evolution of the information contained within this wiki and welcome the addition of content that is beneficial to the community as a whole. However, please DO NOT delete any content from this page!! Significant time, effort, and deliberation has gone into the information contained in this page. Rather than deleting content, please feel free to voice your concerns by posting a comment to the discussion page where others can contribute (please be sure to include a topic to be referenced in responses).

Installing PMF with Igor

Setting up PMF on Your Computer

  1. Download the BareBones PMF Starter Kit
  2. Create a new folder on your computer where you'll store the PMF files and all files output by PMF (this should NOT be the folder where you store your Igor experiments). This folder must contain:
    • PMF2wtst.exe (obtained from P. Paatero or the BareBones Starter Kit)
    • imupmf.ini (obtained from the BareBones Starter Kit)
    • pmf2key.key (obtained from P. Pattero, U. Helsinki)
  3. Start a new Igor experiment and load the following files from the BareBones Starter Kit:
    • DataAndErrorForBareBonesPMF.itx
    • BareBonesPMFExecution_1_00B.ipf


Running the PMF Test Case

  1. Start the Panel from the BareBonesPMF menu on the menu bar.
    File:BareBonesPanel.png

  2. Use the first button and select the path to the folder you created with the PMF2wtst.exe file.
  3. Use the second button to execute PMF.

You should see a black DOS window pop up, scroll a lot out output, and then close. Igor will then tell you, "The PMF barebones analysis was successfully completed within X seconds."

X should be > 0!

If the window goes away immediately and the Igor message says that the analysis was completed within 0 seconds, the execution was not successful. Try the following steps to solve the problem.


If PMF does not Run Properly

1. Look in the folder you created with the PMF2wtst.exe file for the existence of the files

 Matrix.dat and StdDev.dat
If these files exist, Igor was able to access the correct folder. Continue with Step 2.
If these files do not exist, Igor was not able to access the correct folder. Go back to the first button on the panel and check that you've given the correct path to the folder with the PMF2wtst.exe file. Run PMF again by pressing the second button.

2. Look in the folder you created with the PMF2wtst.exe file for the existence of the file

 PMF2.LOG
If this file does not exist, PMF was not run in this folder. Go to step 3.
If this file does exist, PMF attempted to run in this folder. Open the file PMF2.LOG (it is a text file).
Glance down the contents of the file and look for many lines of sequential numbered output, such as
 1 rank1 step chi2=   9282.6     Penalty=  1.5287E+04 Flags GF
 2 rank1 step chi2=   7411.7     Penalty=  1.4084E+04 Flags GF
If these lines are in the file, PMF ran successfully on your computer, which must be fast enough to run this data in less than 1 second. You're done!
If the lines of sequential numerical output are not in the file, you should find the following lines within this file (note that every line is NOT included here, but the lines selected here are in the order they appear in the file):

2a)

 ##PMF2 .ini file for: IMUPMF.INI  --- BareBonesPMF
 Successfully read task initialization file imupmf.ini
 titled:  ##PMF2 .ini file for: IMUPMF.INI  --- BareBonesPMF
If these lines appear in the file, PMF found the .ini file. Continue with Step 2b.
If these lines do not appear, you should see a message about not finding an appropriate .ini file.
  • Make sure that this folder contains the file imupmf.ini, provided with the BareBones PMF Starter Kit. If it did not, copy the file to this folder, delete the file PMF2.LOG and press the second button in the panel again to see whether PMF runs successfully.
If the file imupmf.ini already exists in the same folder as the PMF2wtst.exe file, continue with Step 3.

2b)

 Successfully opened input file      30
 with name MATRIX.DAT
 Successfully opened input file      31
 with name STD_DEV.DAT
If these lines appear in the file, everything should have run correctly. Look at the rest of the PMF2.LOG file to see whether other errors are reported. If you still encounter difficulty, contact Ingrid Ulbrich at <Ingrid dot Ulbrich at colorado dot edu> for assistance and attach the PMF2.LOG file to your email.
If these lines do not appear in the file, you will see a message about PMF not being able to access one of these files. Check that the files are not being used by other programs, delete the file PMF2.LOG, and press the second button on the panel again to see whether PMF runs successfully.

3. Look in the C:\ directory of your computer for the existence of the file

 runpmf.bat
If this file exists, Igor was able to write to your C:\ drive. Continue with Step 3a.
If this file does not appear, Igor was not able to write this file to your C:\ drive. This might be due to high security settings on your computer. You should create a text file with this name (NOT runpmf.bat.txt) and choose to edit it (not open it) with a text editor (e.g., Notepad, WordPad, Emacs, etc.). The file should contain the lines
 cd C:\Documents and Settings\Ingrid Ulbrich\Desktop\
 pmf2wtst imupmf  
NOTE that you must change the path in this example to the path to the folder where you have put the file PMF2wtst.exe!

3a) Execute runpmf.bat by double-clicking on its icon. You should see the black DOS window pop up, scroll output, and close again.

If this happens, PMF has run successfully from the batch. In the Igor experiment, press the second button on the panel to see whether PMF runs successfully.
If the black window does not show scrolled output, continue with Step 3b.

3b) You now need to execute PMF from the command line.

To access the DOS command line, launch the Command Prompt from your Windows Start menu, located in
 Start -> Programs -> Accessories -> Command Prompt
Change from the path displayed in the prompt to the folder where you put the PMF2wtst.exe file. Use the command
 cd
to change directories (e.g.,
 cd Desktop\PMF
). You can change one directory at a time, or several at a time as shown in the example. To move up 1 directory, use
 cd ..	
Be sure that the folder contains the three files
 PMF2wtst.exe (obtained from Paatero or the BareBones Starter Kit)
 imupmf.ini  (obtained from the BaseBones Starter K
 pmf2key.key (obtained from P. Pattero, U. Helsinki)
by listing the contents of the current directory, using the command
 dir	
Then type this command at the prompt to run PMF:
 pmf2wtst imupmf
You should see scrolling output in the command window. The window will not disappear and you can scroll back through the output (which is also saved in the file PMF2.LOG).
If the output contains many lines of sequential numbered output, such as
 1 rank1 step chi2=   9282.6     Penalty=  1.5287E+04 Flags GF
 2 rank1 step chi2=   7411.7     Penalty=  1.4084E+04 Flags GF
PMF ran successfully here. Delete the file PMF2.LOG and in the Igor experiment, press the second button on the panel to whether PMF runs successfully from Igor.
If the output does not contain these lines, go back to Step 2 of this section to examine errors that might be reported by PMF in the file PMF2.LOG.

Creating the Organics and Error Matrices (Step 0)

In the Q-AMS Software (James')

Be sure to use v1.41 or later.

In SQUIRREL

In PIKA

Deleting NaNs/zeros

Recommended Practice: Setting a Minimum Error

Recommended Practice: Downweight "Weak" Variables

Perform PMF Analysis *Step 1*

View PMF Analysis Results *Step 2*

Compare PMF Results with External Factors *Step 3*

Considerations for Choosing a Solution

Other Resources