TBLCOL/Schema Conversion and Its C API

TBLCOL/Schema conversion is done through a translation table. A FITS file is first read into a TBLCOL object, with a transposed row-column order. User then may choose to convert all information in TBLCOL to objects of given schema. Following C routines are provided for this purpose.

  • shCnvTblToSchema
  • shCnvSchemaToTbl
  • Users should be careful when using these routines, as lots of memory checkings are required before these routines can successfully perform the conversion (see Object State Initialization for more details). These checkings are necessary, as memory information is provided at run-time and may be specified on command line in dervish.

    If you are impatient, click here for examples.

    shCnvTblToSchema

    RET_CODE shCnvTblToSchema( TBLCOL* tblCol, /* in: TBLCOL */ void * obj, /* int/out: object or container */ SCHEMATRANS* xtblPtr, /* translation table */ char* schemaName, /* schema name using array */ char** schemaInstAddr, /* array of instance addresses passed in from outside */ int addrNum, /* number of addresses passed in */ int rowIndex, /* row to start for conversion */ int stopIndex, /* row to stop at, 0 = end */ CONTAINER_TYPE contType, /* is schemaPtr a container? what type? */ unsigned short objectReuse /* TRUE if re-use the old objects */ ); where tblCol points to an existing TBLCOL structure obj points to an object to convert to, or points to a container that will hold the objects yet to create. If obj is a container, it is assume to be empty. xtblPtr points to a translation table structure schemaName name of schema for the object pointed to by obj schemaInstAddr array of valid object addresses to use if obj is a container addrNum number of addresses available in schemaInstAddr rowIndex which row of TBLCOL to start from, default is 0 stopIndex which row of TBLCOL to stop at, default is end contType an enum value to specify type of container. Currently the support types are: NOT_A_CONTAINER, ARRAY_TYPE, LIST_TYPE, and CHAIN_TYPE. objectReuse If TRUE, obj should be a pointer to a non-empty container whose objects will be reused. This is useful if one wants to get data from several different TBLCOL structures.

    shCnvSchemaToTbl

    RET_CODE shCnvSchemaToTbl( TBLCOL** tblCol, /* out: tblcol instance to convert to */ void* obj, /* in: schema or array to convert from */ SCHEAMTRANS* xtblPtr, /* in: translation table structure */ char* schemaName, /* schema name that's in array */ CONTAINER_TYPE contType, /* type of container */ unsigned auto_convert, /* TRUE if auto_convert is needed */ ); where tblCol points to the the address of a TBLCCOL* type pointer obj points to an object to convert, or points to a container that holds the objects to convert xtblPtr points to a translation table structure schemaName name of schema for the object pointed to by obj contType an enum value to specify type of container. Currently the support types are: NOT_A_CONTAINER, ARRAY_TYPE, LIST_TYPE, and CHAIN_TYPE. auto_convert If TRUE, elementary types not mentioned in the translation table will be automatically converted.

    Object State Initialization

    Two routines are provided for users to initialize the state of an object.
  • shSpptObjectStateInit
  • shTclSpptObjectStateInit
  • shSpptObjectStateInit executes the object's C constructor. If no constructor is defined, the routine will malloc memory, set elementary types to zero, and malloc memory for pointers. The other function, shTclSpptObjectStateInit, operates similarly. It calls the object's Tcl constructor first (if specified in the translation table) before resorting to the means used by shSpptObjectStateInit. RET_CODE shSpptObjectStateInit( SCHEMATRANS* xtblPtr, /* a translation table */ void* objPtrs, /* objects (not container) */ int objCnt, /* number of objects */ SCHEMA* schemaPtr /* schema of the object */ ); RET_CODE shSpptTclObjectInit( Tcl_Interp* a_interp, /* Tcl Interpretor */ SCHEMATRANS* xtblPtr, /* a translation table */ void** objectPtrs, /* objects (not container) */ int objCnt, /* number of objects */ SCHEMA* schemaPtr, /* schema of the object */ unsigned short handleRetain /* TRUE if wants to retain Tcl handle */ ); Comments on each arguement should be explanatory. All the parameters are input parameters. On return, both routines return SH_SUCCESS if the state of the object (passed in by void* object) is successfully validated.

    Translation Table

    A translation table (type SCHEMATRANS) is basically an array of string associations for run-time string mappings represented by SCHEMATRANS_ENTRY structure. A management structure, SCHEMATRANS, contains an array of SCHEMATRANS_ENTRY. typedef struct _xtable_entry { CONVTYPE type; /* conversion type */ char* src; /* source string -- FITS side*/ char* dst; /* destination string */ char* dsttype; /* dst type string -- schema side*/ char* heaptype; /* heap base type */ char* heaplen; /* heap length string */ DSTTYPE dstDataType; /* destination type */ char* proc; /* malloc command for dst*/ char* size; /* size string */ int num[5]; /* number of blocks to malloc*/ double srcTodst; /* optional ratio of data */ } SCHEMATRANS_ENTRY; typedef struct _xtable { int totalNum; /* total number of entries, empty or not*/ int entryNum; /* current number of good entries */ SCHEMATRANS_ENTRY* entryPtr; /* pointed to entryNum entries */ SCHTR_STATUS * status; /* status of entries */ } SCHEMATRANS; SCHEMATRANS maintains a pointer to totalNum SCHEMATRANS_ENTRYs as well as that many enumerated integers (pointed by status). These status integers record the visit status of each entry when the table is processed.

    The number of entries in a SCHEMATRANS table changes dynamically as users add more entries. When the initial number of entries becomes too small, the translation table can grow automatically to accomodate the new size. Key word MAX_SCHEMATRANS_ENTRY_SIZE specifies the number of empty slots each growth adds to the table. Translation tables can be dynamically generated through this package's Tcl interface.

    Fields in SCHEMATRANS are somewhat self-explanatory. We choose to make src/dst independent of conversion direction by associating src with FITS and dst with SCHEMA so we can use the same translation table for both directions.

    heaptype designates what the base type is when dsttype is "heap", and heaplen is an expression whose results (integer) give the length of heap.

    Proc is a Tcl command string to call when a new object is needed, with optional size information (this command will be called that many times). Users specify their favorite constructor, which will be called first before any other means of malloc is pursued.

    Size is a string to specify size and dimension information. A "5x10" means the field has two dimensions (in addition to whatever is already in schema) with fastest varying dimension having size 10. In other words, it is just like [5][10] in C syntax.

    In practice, it is quite common that physical units of one field in FITS table is different from the one a user's field has. So srcTodst is here to convert the units.

    Other fields in SCHEMATRANS are for internal use and should be of no interest to general users. But briefly, num stores the numerical array size information that hides in the string size and in schema (note, num is 1 dimension has size MAX_INDIRECTION ). Status indicates the status of the given entry where it has been visited, fresh or has good syntax.

    C API for Translation Table

    SCHEMATRANS* shSchemaTransNew(void) creates a new translation table structure and return the pointer to the structure void shSchemaTransDel(SCHEMATRANS* xptr) destroys all the entries in the translation table xptr and then free the memory held by the table structrue. RET_CODE shSchemaTransEntryDel( SCHEMATRANS* xptr, /* translation table */ unsigned int entryNumber, /* entry number to delete */ unsigned short del_related /* if TRUE,delete related entries */ ); Deletes an entry of a translation table pointed by xptr. If del_related is TRUE, all related entries (if a multi-line entry) will be deleted. RET_CODE shSchemaTransEntryAdd( SCHEMATRANS* xptr, /* translation table */ CONVTYPE type, /* conversion type */ char* src, /* string at fits side */ char* dst, /* string at schema side */ char* dsttype, /* schema field type */ char* heaptype, /* heap type */ char* heaplen, /* heap length */ char* proc, /* Tcl procedure to call */ char* size, /* additional size info */ double r, /* units conversion ratio */ int pos /* absolute position to add entry */ /* if -1, add to the end */ ); Add an entry to the translation table. Comments should be explanatory. See schemaTranEntryAddfor more details. Generally speaking, src and dst are the names (strings) to use with FITS and schema, type is the conversion type that can be "name", "cont", or "ignore", dsttype can be "int", "unsigned short", etc. It also can be "heap", in which case one has to specify heaptype and heaplen. Heaplen is a string that'll be evaluated at runtime, such as "region->nrow". Size is a string e.g, "3x2") that gives both dimension and size information for pointer-type fields. This routine dynamically expands the size of the translation table. void shSchemaTransClearEntries(SCHEMATRANS* xptr) removes all the entries in the translation table but retains the memory of the table structure. RET_CODE shSchemaTransEntryImport( SCHEMATRANS* src, /* source translation table */ SCHEMATRANS* dst, /* destination translation table */ int src_start, /* source side: start entry */ int src_end, /* source side: non-inclusive end entry*/ int dst_pos /* dest side: position to add to -1 if add to the end */ ); Performs translation table copying from src to dst. RET_CODE shSchemaTransCreateFromFile( SCHEMATRANS* xptr, /* existing table structure */ char * fileName /* name of the file containing ascii entries */ );

    Examples using the C routines

    If you are thinking about writing your own generic TBLCOL/Schema converter, best examples are the source files ( tclSchema.c and shSchema.c). After seeing these files and knowing that you'd have to take care a lot of petty details, you probably want to think twice.

    On the contrary, writing an application-specific converter is a lot easier. Such need may arise when we frequently want to convert between application-specific FITS files and certain schema.

    Suppose we're to do conversion between TBLCOL and a C structure defined as

    typedef struct tagTest { int a; float c[10]; char* id; REGION** region; } TEST; where REGION is an existing schema. We want to do: FITS side Object Side A <=> a C c ID <=> id NROW <=> region<0>->nrow region<1>->nrow

    TBLCOL to Schema

  • Things known upon entry are: a TBLCOL pointer (filled with FITS data); a SchemaOject or Container that is used to contain the yet-to-create schema objects; and schemaName.
  • Create a translation table by calling shSchemaTransNew() SCHEMATRANS* xptr; char* TheAddressesOfObjects[100]; /* say we'd like to have 100 objects */ LIST* Container=NULL; char tmpstring[20]; xptr = shSchemaTransNew(); if(xptr == NULL) your_error_message_here;
  • Create these objects for(i=0; i < 100; i++) TheAddressesOfObjects[i]=YourConstructorRoutine();
  • Create a container for the 100 objects Container=shListNew(); if(Container==NULL) your_error_message_here;
  • Add to the translation table by a number of entries status = shSchemaTransEntryAdd(xptr, CONVERSION_BY_TYPE, "a", /* will be converted to upper case */ "a", "int", NULL, /* not a heap */ NULL, NULL, /* don't care about Tcl procedure */ NULL, /* no dimension info */ 1.0, /* no units conversion */ -1, /* add at end of the table, i.e, append */ ); if(status != SH_SUCCESS) your_error_message; status = shSchemaTransEntryAdd(xptr, CONVERSION_BY_TYPE, "c", /* will be converted to upper case */ "c", "float", NULL, /* not a heap */ NULL, NULL, /* don't care about Tcl procedure */ NULL, /* no dimension info, array is static */ 1.0, /* no units conversion */ -1, /* add at end of the table, i.e, append */ ); if(status != SH_SUCCESS) your_error_message; status = shSchemaTransEntryAdd(xptr, "name", "id", /* will be converted to upper case */ "id", "string", NULL, /* not a heap */ NULL, NULL, /* don't care about Tcl procedure */ "10", /* assume max strlen(id) is 10 */ 1.0, /* no units conversion */ -1, /* add at end of the table, i.e, append */ ); if(status != SH_SUCCESS) your_error_message; for(i=0; i < 2; i++) { sprintf(tmpstring,"region<%d>", i); status = shSchemaTransEntryAdd(xptr, CONVERSION_BY_TYPE, "regnam", /* will be converted to upper case */ tmpstring, "struct", /* meaning continuation lines follow */ NULL, /* not a heap */ NULL, NULL, /* don't care about Tcl procedure */ "2", /* pointer is equivalent to array of 2 pointers */ 1.0, /* no units conversion */ -1, /* add at end of the table, i.e, append */ ); if(status != SH_SUCCESS) your_error_message; /* continuation line */ status = shSchemaTransEntryAdd(xptr, CONVERSION_CONTINUE, /* continuation line */ tmpstring, "nrow", "int", NULL, /* not a heap */ NULL, NULL, /* don't care about Tcl procedure */ NULL, 1.0, /* no units conversion */ -1, /* add at end of the table, i.e, append */ ); if(status != SH_SUCCESS) your_error_message; }
  • Call syntax check against your schema status = shSpptGeneralSyntaxCheck(xptr, "TEST"); if(status!=SH_SUCCESS) your_error_message;
  • Valid the state of the 100 objects status = shSpptObjectStateInit( xptr, TheAddressesOfObjects, 100, shSchemaGet("TEST")); if(status!=SH_SUCCESS) your_error_message;
  • Pass the pointers to the conversion routine status = shCnvTblToSchema( theTBLCOL, Container, xptr, "TEST", /* our schema name */ TheAddressesOfObjects, 100, 0, /* conversion begins at very 1st row of TBLCOL */ 0, /* convert all */ LIST_TYPE, /* we use LIST as our container */ FALSE, /* new objects */ ); if(status!=SH_SUCCESS) your_error_message;
  • After this call, the fields objects in Container (or the object) have the data given in FITS file.
  • Delete the table and then gracefully return shSchemaTransDel(xptr); return;
  • Schema to TBLCOL

  • Things known upon entry are: TBLCOL pointer (to be filled with data from the object(s)); SchemaObject or Container that contains the schema objects to convert; and schemaName

    Translation tables used for converting schema objects to TBLCOL has exactly the same structure as the ones for TBLCOL-to-schema conversion. In fact, we can use the same table we just created in the previous example.

  • Create and fill a translation table by calling shSchemaTransNew() and shSchemaTransEntryAdd() TBLCOL* tblcolPtr = NULL; /* a pointer that will be set to point to the output tblcol */ CHAIN* Container; /* suppose this pointer points to the chain container that holds a set of TEST objects to convert */ SCHEMATRANS* xptr; xptr = shSchemaTransNew(); if(xptr == NULL) your_error_message_here; /* See the previous example for details ... */ /* ........ */
  • Call syntax check against your schema status = shSpptGeneralSyntaxCheck(xptr, "TEST"); if(status!=SH_SUCCESS) your_error_message;
  • Pass the pointers to the conversion routine status = shCnvSchemaToTbl( &tblcolPtr, Container, xptr, "TEST", /* our schema name */ CHAIN_TYPE, /* our container type */ TRUE /* yes! we want to auto-convert fields unmentioned in the translation table */ ); if(status!=SH_SUCCESS) your_error_message;
  • After this call, TBLCOL is filled with the data from the object(s).
  • Delete the table and then gracefully return shSchemaTransDel(xptr); return;

  • Wei Peng