The procedures of interest are:
test-1.fits test-2.fits ... test-10.fitsIf you would like to process this sequence of files with an overlap of 10 rows, then the frame size would be 110 (i.e., number of rows in FITS file plus number of overlap rows). The following TCL line would initialize the state:
dervish> set state [nextFrameInit . test- 1 10 .fits 10 110 0]Then the nextFrameGet and nextEOF proc's could be used to read each FITS file in the sequence and determine when the last file has been read:
dervish> set region [keylget state frameHandle] dervish> set state [nextFrameGet $state] dervish> if { ![nextEOF state] } then {...do something with $region...}A loop in a TCL script to do this might look like:
set state [nextFrameInit . test- 1 10 .fits 10 110 0] set region [keylget state frameHandle] # Save frame region handle set state [nextFrameGet $state] # Get first frame while { ![nextEOF $state] } { # Loop over FITS file sequence ...do something with $region... set state [nextFrameGet $state] # Get next frame }
nextFrameInit <dirName> <baseName> <beginFile> <endFile> <suffixName> <overlapRows> <frameRows> <nfDebug> <dirName> Directory containing FITS files <baseName> Base component of file names <beginFile> Beginning (numbered) component of file name <endFile> Ending (numbered) compenent of file name <suffixName> Suffix of file names <overlapRows> Number of rows in overlap <frameRows> Number of rows in frame (FITS file rows + overlap rows) <nfDebug> 0 = No debug statements; 1 = Print debug statements
nextFrameGet <state> <state> Frame state
nextEOF <state> <state> Frame state
set fieldState [nextFieldInit <dirName> <filterList> <baseName> <beginFile> <endFile> <suffixName> <overlapRows> <frameRows> <nfDebug>] <filterList> list of filter names, in the order they appear on the tape <dirName> Directory containing FITS files <baseName> Base component of file names <beginFile> Beginning (numbered) component of file name <endFile> Ending (numbered) compenent of file name <suffixName> Suffix of file names <overlapRows> Number of rows in overlap <frameRows> Number of rows in frame (FITS file rows + overlap rows) <nfDebug> 0 = No debug statements; 1 = Print debug statements
set fieldState [nextFieldGet <fieldState>] <fieldState> Field state
regFromFilter <filter> <fieldState>
proc nextFieldGetExample {dirName iRun iCol} { set baseName run[format %06d $iRun]-c$iCol set hdr [hdrNew] set infoChain [fits2Schema $dirName/$baseName.fieldInfo FI $hdr] set filterList [hdrGetAsAscii $hdr FILTERS] echo call initNextField: set iField 0 set debug 1 set fieldState [nextFieldInit $filterList \ $dirName $baseName- 0 1000 .fits 200 1554 $debug] echo ============================================================ echo MemStatsPrint after nextFieldInit: echo [memStatsPrint] echo ============================================================ echo [timerStart] loop iField 0 7 { # Get the next field echo call nextFieldGet for iField = $iField set fieldState [nextFieldGet $fieldState] echo ============================================================ echo MemStatsPrint after nextFieldGet: echo [memStatsPrint] echo ============================================================ # Do the analysis set memPt [expr 1+[memSerialNumber]] foreach filter $filterList { echo - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - set reg [regFromFilter $filter $fieldState] echo for filter=$filter and reg=$reg set ccdRow [hdrGetAsInt $reg.hdr CCDROW] set ccdCol [hdrGetAsInt $reg.hdr CCDCOL] set frame [hdrGetAsInt $reg.hdr FRAME] echo ccdRow=$ccdRow ccdCol=$ccdCol frame=$frame set if [chainElementGetByPos $infoChain $iField] echo Information from the infoChain: exprPrint $if handleDel $if set reg [regNew 1000 1000] set reg [regNew 1000 1000] set reg [regNew 1000 1000] set reg [regNew 1000 1000] set reg [regNew 1000 1000] } echo -----------> Free up the memory allocated during iField=$iField memFreeBlocks $memPt [memSerialNumber] echo [timerLap] } return 0 }And here is its output:
al> nextFieldGetExample /data/dp1.3/astroline/run/601Input 601 3 call initNextField: --- bufferFill: thisFile=0 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000000.fits --- bufferFill: thisFile=1 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000001.fits --- bufferFill: thisFile=2 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000002.fits --- bufferFill: thisFile=3 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000003.fits --- bufferFill: thisFile=4 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000004.fits ============================================================ MemStatsPrint after nextFieldInit: Number of memory allocation requests: 504 Number of memory de-allocation requests: 82 Total bytes currently in use: 103575705 Total bytes in Free Memory Pool: 9472 Percentage of memory allocation requests satisfied from Free Memory Pool: 13.10 % Percentage of memory allocation requests satisfied from the Operating System: 86.90 % ============================================================ call nextFieldGet for iField = 0 --- bufferFill: thisFile=5 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000005.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=6 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000006.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=7 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000007.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=8 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000008.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=9 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000009.fits --- nextEOF: No EOF yet ============================================================ MemStatsPrint after nextFieldGet: Number of memory allocation requests: 1164 Number of memory de-allocation requests: 492 Total bytes currently in use: 103617170 Total bytes in Free Memory Pool: 19200 Percentage of memory allocation requests satisfied from Free Memory Pool: 40.81 % Percentage of memory allocation requests satisfied from the Operating System: 59.19 % ============================================================ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=r and reg=h5 ccdRow=1 ccdCol=3 frame=15 Information from the infoChain: lambda -2.89377777 eta 0 time 50893.3391839 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=i and reg=h8 ccdRow=2 ccdCol=3 frame=17 Information from the infoChain: lambda -2.89377777 eta 0 time 50893.3391839 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=u and reg=h11 ccdRow=3 ccdCol=3 frame=19 Information from the infoChain: lambda -2.89377777 eta 0 time 50893.3391839 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=z and reg=h14 ccdRow=4 ccdCol=3 frame=21 Information from the infoChain: lambda -2.89377777 eta 0 time 50893.3391839 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=g and reg=h17 ccdRow=5 ccdCol=3 frame=23 Information from the infoChain: lambda -2.89377777 eta 0 time 50893.3391839 -----------> Free up the memory allocated during iField=0 {ELAPSED 40.881} {CPU {{OVERALL 30.970} {UTIME 25.360} {STIME 5.610} {CUTIME 0.000} {CSTIME 0.000}}} call nextFieldGet for iField = 1 --- bufferFill: thisFile=10 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000010.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=11 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000011.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=12 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000012.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=13 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000013.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=14 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000014.fits --- nextEOF: No EOF yet ============================================================ MemStatsPrint after nextFieldGet: Number of memory allocation requests: 1984 Number of memory de-allocation requests: 1312 Total bytes currently in use: 103617170 Total bytes in Free Memory Pool: 50138240 Percentage of memory allocation requests satisfied from Free Memory Pool: 59.17 % Percentage of memory allocation requests satisfied from the Operating System: 40.83 % ============================================================ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=r and reg=h5 ccdRow=1 ccdCol=3 frame=16 Information from the infoChain: lambda -2.59288889 eta 0 time 50893.3400198 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=i and reg=h8 ccdRow=2 ccdCol=3 frame=18 Information from the infoChain: lambda -2.59288889 eta 0 time 50893.3400198 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=u and reg=h11 ccdRow=3 ccdCol=3 frame=20 Information from the infoChain: lambda -2.59288889 eta 0 time 50893.3400198 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=z and reg=h14 ccdRow=4 ccdCol=3 frame=22 Information from the infoChain: lambda -2.59288889 eta 0 time 50893.3400198 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=g and reg=h17 ccdRow=5 ccdCol=3 frame=24 Information from the infoChain: lambda -2.59288889 eta 0 time 50893.3400198 -----------> Free up the memory allocated during iField=1 {ELAPSED 77.152} {CPU {{OVERALL 58.760} {UTIME 49.790} {STIME 8.970} {CUTIME 0.000} {CSTIME 0.000}}} call nextFieldGet for iField = 2 --- bufferFill: thisFile=15 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000015.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=16 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000016.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=17 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000017.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=18 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000018.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=19 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000019.fits --- nextEOF: No EOF yet ============================================================ MemStatsPrint after nextFieldGet: Number of memory allocation requests: 2804 Number of memory de-allocation requests: 2132 Total bytes currently in use: 103617170 Total bytes in Free Memory Pool: 50138240 Percentage of memory allocation requests satisfied from Free Memory Pool: 71.11 % Percentage of memory allocation requests satisfied from the Operating System: 28.89 % ============================================================ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=r and reg=h5 ccdRow=1 ccdCol=3 frame=17 Information from the infoChain: lambda -2.292 eta 0 time 50893.3408556 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=i and reg=h8 ccdRow=2 ccdCol=3 frame=19 Information from the infoChain: lambda -2.292 eta 0 time 50893.3408556 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=u and reg=h11 ccdRow=3 ccdCol=3 frame=21 Information from the infoChain: lambda -2.292 eta 0 time 50893.3408556 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=z and reg=h14 ccdRow=4 ccdCol=3 frame=23 Information from the infoChain: lambda -2.292 eta 0 time 50893.3408556 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=g and reg=h17 ccdRow=5 ccdCol=3 frame=25 Information from the infoChain: lambda -2.292 eta 0 time 50893.3408556 -----------> Free up the memory allocated during iField=2 {ELAPSED 123.107} {CPU {{OVERALL 86.720} {UTIME 74.190} {STIME 12.530} {CUTIME 0.000} {CSTIME 0.000}}} call nextFieldGet for iField = 3 --- bufferFill: thisFile=20 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000020.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=21 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000021.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=22 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000022.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=23 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000023.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=24 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000024.fits --- nextEOF: No EOF yet ============================================================ MemStatsPrint after nextFieldGet: Number of memory allocation requests: 3624 Number of memory de-allocation requests: 2952 Total bytes currently in use: 103617170 Total bytes in Free Memory Pool: 50138240 Percentage of memory allocation requests satisfied from Free Memory Pool: 77.65 % Percentage of memory allocation requests satisfied from the Operating System: 22.35 % ============================================================ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=r and reg=h5 ccdRow=1 ccdCol=3 frame=18 Information from the infoChain: lambda -1.99111111 eta 0 time 50893.3416914 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=i and reg=h8 ccdRow=2 ccdCol=3 frame=20 Information from the infoChain: lambda -1.99111111 eta 0 time 50893.3416914 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=u and reg=h11 ccdRow=3 ccdCol=3 frame=22 Information from the infoChain: lambda -1.99111111 eta 0 time 50893.3416914 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=z and reg=h14 ccdRow=4 ccdCol=3 frame=24 Information from the infoChain: lambda -1.99111111 eta 0 time 50893.3416914 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=g and reg=h17 ccdRow=5 ccdCol=3 frame=26 Information from the infoChain: lambda -1.99111111 eta 0 time 50893.3416914 -----------> Free up the memory allocated during iField=3 {ELAPSED 172.125} {CPU {{OVERALL 114.380} {UTIME 98.570} {STIME 15.810} {CUTIME 0.000} {CSTIME 0.000}}} call nextFieldGet for iField = 4 --- bufferFill: thisFile=25 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000025.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=26 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000026.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=27 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000027.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=28 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000028.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=29 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000029.fits --- nextEOF: No EOF yet ============================================================ MemStatsPrint after nextFieldGet: Number of memory allocation requests: 4444 Number of memory de-allocation requests: 3772 Total bytes currently in use: 103617170 Total bytes in Free Memory Pool: 50138240 Percentage of memory allocation requests satisfied from Free Memory Pool: 81.77 % Percentage of memory allocation requests satisfied from the Operating System: 18.23 % ============================================================ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=r and reg=h5 ccdRow=1 ccdCol=3 frame=19 Information from the infoChain: lambda -1.69022222 eta 0 time 50893.3425272 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=i and reg=h8 ccdRow=2 ccdCol=3 frame=21 Information from the infoChain: lambda -1.69022222 eta 0 time 50893.3425272 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=u and reg=h11 ccdRow=3 ccdCol=3 frame=23 Information from the infoChain: lambda -1.69022222 eta 0 time 50893.3425272 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=z and reg=h14 ccdRow=4 ccdCol=3 frame=25 Information from the infoChain: lambda -1.69022222 eta 0 time 50893.3425272 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=g and reg=h17 ccdRow=5 ccdCol=3 frame=27 Information from the infoChain: lambda -1.69022222 eta 0 time 50893.3425272 -----------> Free up the memory allocated during iField=4 {ELAPSED 201.786} {CPU {{OVERALL 141.560} {UTIME 122.850} {STIME 18.710} {CUTIME 0.000} {CSTIME 0.000}}} call nextFieldGet for iField = 5 --- bufferFill: thisFile=30 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000030.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=31 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000031.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=32 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000032.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=33 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000033.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=34 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000034.fits --- nextEOF: No EOF yet ============================================================ MemStatsPrint after nextFieldGet: Number of memory allocation requests: 5264 Number of memory de-allocation requests: 4592 Total bytes currently in use: 103617170 Total bytes in Free Memory Pool: 50138240 Percentage of memory allocation requests satisfied from Free Memory Pool: 84.61 % Percentage of memory allocation requests satisfied from the Operating System: 15.39 % ============================================================ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=r and reg=h5 ccdRow=1 ccdCol=3 frame=20 Information from the infoChain: lambda -2.74333334 eta 0 time 50893.3396019 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=i and reg=h8 ccdRow=2 ccdCol=3 frame=22 Information from the infoChain: lambda -2.74333334 eta 0 time 50893.3396019 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=u and reg=h11 ccdRow=3 ccdCol=3 frame=24 Information from the infoChain: lambda -2.74333334 eta 0 time 50893.3396019 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=z and reg=h14 ccdRow=4 ccdCol=3 frame=26 Information from the infoChain: lambda -2.74333334 eta 0 time 50893.3396019 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - for filter=g and reg=h17 ccdRow=5 ccdCol=3 frame=28 Information from the infoChain: lambda -2.74333334 eta 0 time 50893.3396019 -----------> Free up the memory allocated during iField=5 {ELAPSED 228.413} {CPU {{OVERALL 167.250} {UTIME 146.940} {STIME 20.310} {CUTIME 0.000} {CSTIME 0.000}}} call nextFieldGet for iField = 6 --- bufferFill: thisFile=35 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000035.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=36 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000036.fits --- nextEOF: No EOF yet --- bufferFill: thisFile=37 endFile=1000 --- bufferFill: Reading /data/dp1.3/astroline/run/601Input/run000601-c3-000037.fits --- nextEOF: No EOF yet