sdsscompress [-Vvacmf] [-b bits] [file...] sdssdecompress [-Vvcf] [file...]OPTIONS
-V Show version -v Verbose - show input and out file names and the ratio of output to input file size -a Show compression estimate calculations -c Pass compressed/uncompressed output to stdout -m Use the mode estimated in the first block to produce comma codes. Default is to use the difference from the previous pixel. compress only -f Force overwrite of output file if estimated or actual compressed size are not less that the input size or if the files already exist. If the program is run in background -f is the default. In foreground with out -f the user is prompted about overwriting files. -b Force b significant bit coding. compress only
The input may be either stdin (default) or a list of files. If it is stdin then compressed output is written to stdout. If it is a list of files then each of the files is rewritten compressed, a .R is appended to the file name, and the input file is deleted. So, there must be room for both copies of the largest file.
If the estimated or actual compression are larger than the input file the output file is not written. You can force the writing of the output file with the -f option. This should not normally be used because sdssdecompress filename checks for a filename.R file for its input.
sdssdecompress input may be either stdin or a list of files. If it is stdin then compressed output is written to stdout. If it is a list of files then each of the files is rewritten compressed. If a file in the list of files does not have a trailing .R then if there is a file with a trailing .R it is used as the put file. The .R is removed from the output file name.
The output file maintains the FITS header, permission modes, and access and modification times of the input file.
The FITS header is copied in 2880 byte blocks with the last block determined by "END " in columns 1-9 of a 80 character header record.
During compression if the value in column 10 of the "SIMPLE" record is a 'T' it is changed to '1' and if it is a 'F' it is changed to '0'. This makes the file non Fits standard. During decompression those values are restored.
The compression version number and number of compression bits are written in the first two bytes after the FITS header in compressed files.
The algorithm is based on an algorithm suggested by Robert F. Rice and James R. Plaunt in "Adaptive Variable-Length Coding for Efficient Compression of Spacecraft Television Data"; IEEE Transactions on Communication Technology; Vol 19, No 6, December 1971.
Pixels are split into the n most significant bits, MSB, which are assumed to be Laplacian distributed and the 16-n least significant bits, LSB, which are assumed to be uniformly distributed so they have very high entropy and contribute nothing to the compression.
The program uses a comma code to compress the MSB of the pixels. The compressed value is the difference between the pixel and the previous pixel or if the -m option is specified between the pixel and estimated mode of the data. The mode of the file is estimated from the first 8192 pixels.
The comma codes are as follows:
difference bit code 0 1 1 01 -1 001 2 0001 -2 00001 3 000001 -3 0000001 4 00000001 -4 000000001 5 0000000001 -5 00000000001 outlyer 000000000001 6 0000000000001 -6 00000000000001 7 000000000000001 -7 0000000000000001 8 00000000000000001 -8 000000000000000001 9 0000000000000000001 -9 00000000000000000001 10 000000000000000000001 -10 0000000000000000000001 11 00000000000000000000001 -11 000000000000000000000001 12 0000000000000000000000001 -12 00000000000000000000000001Outlyers with an absolute difference greater than 12 are coded with a comma code equal to 10 followed by the actual pixal.
The program estimates the number of compressable bits in the data from the first 8192 calculating the compression compression using 16 through 1 MSB until the compressing increases. It uses the number of MSB that produces the best estimated compression.
The first example compresses a file and then decompresses it:
SDSSDP2> sdsscompress idR-000581-31-0015.fit SDSSDP2> ls -l idR-000581-31-0015.fit* -rw-r--r-- 1 kurt system 1708491 Dec 11 19:13 idR-000581-31-0015.fit.R SDSSDP2> sdssdecompress idR-000581-31-0015.fit SDSSDP2> ls -l idR-000581-31-0015.fit* -rw-r--r-- 1 kurt system 5768640 Dec 11 19:13 idR-000581-31-0015.fit
This example attempts to compress two files. The first is successfully compressed and is given a .R suffix. The second does not obtain smaller compression so is not rewritten. The decompression attempts to do both files. The first is successfully decompressed. The second since it was never compressed is also not decompressed. The -v options prints the compression ratio.
SDSSDP2> sdsscompress -v z.fits uncompressable.fits z.fits: 26.88% -- replaced with z.fits.R uncompressable.fits: 137.00% -- Negative compression, file unchanged SDSSDP2> sdssdecompress -v z.fits uncompressable.fits z.fits.R: SDSS compress version 1 compressed with 2 LSB 374.21% -- replaced with z.fits uncompressable.fits.R: No such file or directory
This example uses the -c option to compress a file to stdout; decompresses stdout; and compares that result to the original.
SDSSDP2: sdsscompress -c bigdata.fits \ > | sdssdecompress | cmp - bigdata.fits
0 Successful compression/decompression 1 An error occurred and a message is printed 2 The compression did not reduce the size of the file
The programs report the following errors:
not a regular file: unchanged The input file is not a unix data file. Return status 1. has %d other links: unchanged The input file has links pointing to it and is not removed. Return status 1. fits file invalid naxis The input file does not have NAXIS = 2 in the fits header. Return status 1. fits file invalid bitpix The input file does not have BITPIX = 16 in the fits header. Return status 1. Negative compression, file unchanged The estimated or actual compression ratio is greater than 1. Return status 2.