Odd/Even Pipeline

The purpose of this pipeline is to correct raw imager frames in Camcol 1 for problems with glitches in the bias voltages. Normally, frames from the g' and r' CCDs have a strong modulation in the bias level as a function of the odd/even ccd column number. Occassionally, however, the camera electronics transition to a state where the pattern disappears; the mean bias level also changes. The pipeline functions as follows:

  1. For the g' and r' CCDs in camcol 1, read the bias vectors and compute the mean difference odd-even in the overscan columns
  2. For all g' and r' files in camcol 1, compute the median bias level in each ccd column in the overscan regions (cols 0-39 and 2108-2127); and compute the mean odd-even difference. If this is different from the bias frames, write the file name to a file.
  3. Copy all original idR files to a destination directory where file fixes will be done. Using the file created in the previous step, reprocess each file and construct a vector that is subtracted from the frame such that when the bias vector is subsequently subtracted by photo, no residual pattern remains.
  4. For the files where the electronics transition in the middle of the frame, a special routine is run to correct the top and bottom sections of each frame separately; the dividing row is input by the user


Top level commands:

  • Preparation: A typical directory structure might be:

    			/data/dp3.b/data
    				|
    				|___________________________
    				|			    |
    			      1412			    |
    				|			    |
    				|			    |
    	Rerun			1			standards
    		                |			    |
    		                |			  calib
    			      fields			    |
    		                |			   1400
    		 	 _______|_______			    |
    		        |       	|		   idB...
    	Camcol        1.orig		1
    			|		|
    	              idR...           idR...
    

    Two files must be prepared for input: oePlan.par and qc.par. A typical oePlan.par file looks like:

    dataDir /data/dp2.f/data/1412/1/fields/1.orig
    outDir /data/dp2.f/data/1412/1/fields/1
    biasDir  /data/dp3.b/data/standards/calib/1400
    thresh 10
    gshift -8.2
    rshift -5.4
    

    A typical qc.par file looks like:

    oe_thresh 10
    

  • oddEvenCheckJob oePlan.par qc.par: Runs through the original idR files (in dataDir) and matches the odd/even pattern in each against the bias. Names of files that need correcting are written to "parameters.par" that is deposited in the same directory as "oePlan.par". Files that might need correction because the bias levels change in mid-frame are printed to "oeTransition.par" in the same directory as "oePlan.par". The number of good and bad files is printed. If nbad = 0, the remaining steps can be skipped. The return code is 0 if all frames are good, 1 otherwise.

  • oddEvenJob oePlan.par: This step should be run if the return code from oddEvenCheckJob is 1. Copy all original idR files to the new rerun directory (fields/1). Read "parameters.par" and fix those files that need correction. The return code should always be 0.

  • oddEvenSplitJob oePlan.par file row: This job is optional and is normally run by hand, since the transition row cannot be determined automatically. Process a file where the bias level changed in mid-frame. "file" is the name of the file (without directory info). "row" is the approximate row where the bias transition occurred, as determined from visual inspection of the frame. Note that sometime the transition occurs by bouncing around over several rows; in this case, one should just pick a row number somewhere in the middle of the transition region. If this job is run, the oeTransitions.par file should be deleted (since all frames are now good).

  • oddVerifyCheckJob oePlan.par qc.par qcfile: Runs through the copied and repaired idR files and matches the odd/even pattern in each against the bias. An output file qcfile is created with a report. The number of good and bad files is printed. If nbad = 0, the oddEven fix worked. The return code is 0 if all frames are good, 1 otherwise. A return code of 1 means something very bad happened, and help should be sought.
  • Note: the bias transitions occur at the same time in the g' and r' CCDs. Because the idR files are numbered by field (which are not read at the same time in all CCDs), the g' and r' fields that correspond to the same time are separated by 8 fields (e.g., g' field 168 matches r' field 176).

    Note: The remaining CCD frames (u', i', z') also show a small bias level shift due to the bias transition. This shift is not corrected, and only affects the frames in the transition region.