FITS I/O and Tables

TCL API

C Routine Interface

Contributed Functions

FITS Tables Under Dervish

TBLCOLs are defined in the header file shCTbl.h.

A generic FITS ASCII and Binary Table reader is supported by Dervish. Both table types are described in NOST's (NASA/Science Office of Standards and Technology) Definition of the Flexible Image Transport System (FITS) standards document (NOST 100-1.1, September 29, 1995) and A User's Guide for the Flexible Image Transport System (FITS) (Version 3.0, October 1992)

NOTE: A later version of the User's Guide is available (Version 3.1, May 2, 1994). The new Binary Table substring array convention is not supported. Although not related to Tables, the IMAGE extension is also not supported (see Section 5.3.1, "IMAGE Extension", (page 70) of the newer User's Guide).

Under Dervish, ASCII and Binary Tables are stored in-memory in what is called the columnar, TBLCOL, format. When refering to the term "ASCII" or "Binary" Table, Dervish usually means the in-memory copy, not the header and data unit (HDU) in the FITS file. The context will clarify which Table, in-memory or FITS, is being discussed.

The TBLCOL format is flexible enough to accomodate both ASCII and Binary Tables, including heap from Binary Tables. Once a Table is created, there is little need for the user to differentiate between an ASCII or a Binary Table (except when it is written out to a FITS file). The TBLCOL format allows the:

The TBLCOL format stores ASCII or Binary Table fields in arrays, each array corresponding to a field. This allows quick and easy retrieval of data in a column (field) oriented way. An ASCII or Binary Table may be read into memory even if the user has no a priori knowledge of the FITS file contents or Table structure.

To access Table data on a row-by-row basis more efficiently, a TBLCOL Table can be copied to a Dervish handle and object schema. This allows access to Table data through C-like structures. This format is more appropriate when accessing Table data row by row, using multiple fields from the same row (rather than accessing all values of one field which the TBLCOL format facilitates). But, unlike the TBLCOL format, the Dervish handle must be constructed by the user. And, the handle is not considered to be a Table.

The following interfaces are provided to create and access Dervish Tables:

  • ANSI C
  • Tcl
  • The following interfaces are provided to read and write FITS ASCII and Binary Tables:
  • ANSI C
  • Tcl
  • Dervish vs. FITS

    0-indexing vs. 1-indexing

    Dervish and FITS index structures, arrays, rows, fields, etc. differently. Dervish uses 0-indexing. Under Dervish, FITS' 1-indexing is changed to 0-indexing. For example, the Primary Header and Data Unit (PDU) is the 1st HDU (Header and Data Unit) under FITS, but the HDU 0 under Dervish. Likewise, the 1st field in a Table is indexed with 0 under Dervish.

    Still, Dervish error messages may indicate positions with oridinal numbers, but this is made clear through the use of suffixes such as st, nd, rd, and th.

    There may still be some confusion, since FITS Table header keywords are still 1-indexed. When Dervish 0-indexes a field with f, the corresponding FITS header keyword is KEYWORDn where n is (f + 1).

    Row-major vs. Column-major Order

    Dervish uses C's convention of referencing arrays in row-major order while FITS uses FORTRAN's convention of referencing arrays in column-major order. Consider the example (shown 0-indexed with n elements)

         (d , d , ..., d   )
           0   1        n-1
    
    Under row-major ordering, the fastest varying index is the last index, namely
         d
          n-1
    
    Under column-major ordering, the fastest varying index is the first index, namely
         d
          0
    

    When reading in FITS files, Dervish will change array dimensions from column-major to row-major. When an in-memory Table is written out in FITS format, the row-major ordering will be changed to column-major for compatibility with the FITS standard.

    What Comes In is not What Goes Out

    In the case where a Dervish Table is created by reading it from a FITS ASCII or Binary Table from a FITS file, Dervish Tables are different than their originating FITS Table. They are not an exact in-memory copy of the original HDU. The data content is the same to the extent that a binary form of the original data can represent (this is especially true for numeric values from ASCII Tables).

    But, there is also some loss of information, namely comments that existed in the FITS header, but were not carried over to the Dervish header or maintained in the TBLCOL structures. When a Dervish Table is written out to a FITS file HDU, it will probably not be an exact copy of the FITS HDU that it originated from.