Odd/Even Pipeline
The purpose of this pipeline is to correct raw imager frames in Camcol 1
for problems with glitches in the bias voltages. Normally, frames
from the g' and r' CCDs have a strong modulation in the bias level
as a function of the odd/even ccd column number. Occassionally, however,
the camera electronics transition to a state where the pattern
disappears; the mean bias level also changes. The pipeline functions as
follows:
- For the g' and r' CCDs in camcol 1, read the bias vectors and compute
the mean difference odd-even in the overscan columns
- For all g' and r' files in camcol 1, compute the median bias level
in each ccd column in the overscan regions (cols 0-39 and 2108-2127);
and compute the mean odd-even difference. If this is different from
the bias frames, write the file name to a file.
- Copy all original idR files to a destination directory where file fixes
will be done.
Using the file created in the previous step, reprocess each file
and construct a vector that is subtracted from the frame such that
when the bias vector is subsequently subtracted by photo, no
residual pattern remains.
- For the files where the electronics transition in the middle of the
frame, a special routine is run to correct the top and bottom sections
of each frame separately; the dividing row is input by the user
Top level commands:
Preparation: A typical directory structure might
be:
/data/dp3.b/data
|
|___________________________
| |
1412 |
| |
| |
Rerun 1 standards
| |
| calib
fields |
| 1400
_______|_______ |
| | idB...
Camcol 1.orig 1
| |
idR... idR...
Two files must be prepared for input: oePlan.par and qc.par.
A typical oePlan.par file looks like:
dataDir /data/dp2.f/data/1412/1/fields/1.orig
outDir /data/dp2.f/data/1412/1/fields/1
biasDir /data/dp3.b/data/standards/calib/1400
thresh 10
gshift -8.2
rshift -5.4
A typical qc.par file looks like:
oe_thresh 10
oddEvenCheckJob oePlan.par qc.par: Runs through
the original idR files (in dataDir)
and matches the odd/even pattern in each against
the bias. Names of files that need correcting are written to
"parameters.par" that is deposited in the same directory as "oePlan.par".
Files that might need correction because the bias
levels change in mid-frame are printed to "oeTransition.par" in the same
directory as "oePlan.par".
The number of good and bad files is printed.
If nbad = 0, the remaining steps can be skipped. The return code is 0
if all frames are good, 1 otherwise.
oddEvenJob oePlan.par: This step should be run if the
return code from oddEvenCheckJob is 1. Copy all original idR files to the
new rerun directory (fields/1). Read "parameters.par" and fix those files
that need correction. The return code should always be 0.
oddEvenSplitJob oePlan.par file row:
This job is optional and is normally run by hand, since the transition
row cannot be determined automatically.
Process a file where the bias level changed in mid-frame. "file" is
the name of the file (without directory info). "row" is the approximate
row where the bias transition occurred, as determined from visual
inspection of the frame. Note that sometime the transition occurs
by bouncing around over several rows; in this case, one should just
pick a row number somewhere in the middle of the transition region.
If this job is run, the oeTransitions.par file should be deleted (since
all frames are now good).
oddVerifyCheckJob oePlan.par qc.par qcfile:
Runs through
the copied and repaired idR files and matches the odd/even pattern
in each against the bias. An output file qcfile is created with
a report.
The number of good and bad files is printed.
If nbad = 0, the oddEven fix worked. The return code is 0
if all frames are good, 1 otherwise. A return code of 1 means something
very bad happened, and help should be sought.
Note: the bias transitions occur at the same time in the g' and r' CCDs.
Because the idR files are numbered by field (which are not read at the
same time in all CCDs), the g' and r' fields that correspond to the same
time are separated by 8 fields (e.g., g' field 168 matches r' field 176).
Note: The remaining CCD frames (u', i', z') also show a small bias level
shift due to the bias transition. This shift is not corrected, and
only affects the frames in the transition region.