Procedures for creating, accessing, and manipulating Dervish Tables are provided:
Users should be careful to understand the differences in nomencalture between Dervish and FITS.
Tcl commands referencing a particular Table field can specify one of two field locating criteria: search by position or search by name. The -col option indicates the selected field by position. The position value is 0-indexed.
The -field option selects a field by its name.
Not all Table fields have a name associated with them. Those that do,
store that name in the TBLFLD
member TTYPE
.
By default, the first field that matches the -field name exactly
is the selected field.
This behaviour can be changed.
-nocase indicates that the comparisons of the -field
name against the TTYPE
names will be case insensitive
(-case can be used to explicitly indicate that these comparisons
are case sensitive).
Also, -fieldpos n can be used to indicate that the nth
field (where n is 0-indexed) matching the -field name
be selected.
Create a Table (TBLCOL structure) and return a new handle to point to the Table. tblColNew is an alias for the following command:
handleNewFromType TBLCOL
Delete the Table (TBLCOL structure) and its data pointed to by a handle. tblColDel is an alias for the following command:
handleDelFromType(which in turn invokes shTblcolDel).
Return information about the TBLCOL Table. The result is a Tcl keyed list:
{ROWCNT 5} {FLDCNT 10} ...The following keys are always returned:
Add a new field to an existing Table. The added field will be the last field in the Table. Space for data within the field will also be allocated. If heap storage is specified (with the -heaptype option), space for the variable length data to be contained in the heap is allocated also. In both cases, the data area is zeroed.
Many of the switches correspond to the keyed data returned by tblFldInfoGet. The data type contained in the field, -type and/or -heaptype, must be specified. The data type must be one of the data types supported through object schemas. For example, some of the primitive data types are SCHAR, UCHAR, INT, UINT, SHORT, USHORT, LONG, ULONG, FLOAT, DOUBLE, and STR (character strings). A distinction should be made between character (signed or unsigned) values and logical values (the LOGICAL object schema). The UNKNOWN object schema is not permitted.
The field data can be an array. Implicitly, a field is a 1-dimensional array, whose size is the number of rows in the Table. A multi-dimensional array can be specified through the -dim option. The dimensions are in a list specified in row-major order, unlike FITS' column-major order for TDIMn keywords. Also, the first (slowest varying) dimension must be the number of rows in the Table (equivalent to NAXIS2) (which is not present in the FITS file TDIMn keyword for Binary Table extensions). This can be zero (0) (in effect, an empty Table with fields). If the TBLCOL is empty (no fields), then -dim must be explicitly specified to establish the Table's row count.
Data indirection (corresponding to tblFldInfoGet's NSTAR keyword) is not supported. The field's ARRAY contains all the data. This is even true for heap data, as the ARRAY contains TBLHEAPDSC heap descriptors.
The data type
STR
should be used to represent character strings, rather than an array of
CHAR.
Using STR mandates the use of -dim. Besides the
number of Table rows (the first dimension), the maximum size of the string
must be specified (it's possible to have multi-dimensional strings also).
Strings should be null-terminated, so the last dimension size (fastest
varying index) should accomodate the
terminator
expected for guaranteed null-termination.
Heap data can also be specified.
If -heaptype is specified, then heap data will be used.
-type must be
TBLHEAPDSC
(-type can be omitted, in which case it will be defaulted properly).
-heaptype can be any object schema that -type allows,
except TBLHEAPDSC.
As for the main data ARRAY, character strings (STR)
should have an additional character used for guaranteed null-termination.
The field's ARRAY contains heap descriptors
(TBLHEAPDSC) rather than the actual data.
The ARRAY must be 1-dimensional, thus -dim may only
specify the number of Table rows.
The heap data for each Table row is also 1-dimensional, but variable in size.
The lengths of the heap data are specified through -heapcnt.
Although all heap data does not need to be allocated,
heap data lengths must be specified in a list for each Table row,
even if the remaining lengths in the list are all zero (indicating no space
allocation).
If the heap data type is STR, the lengths should reflect the
additional character
used to guarantee a null-terminator.
Based on this list of lengths, heap space is allocated and the
ARRAY's heap descriptors (TBLHEAPDSC) are linked to
the heap.
Return information about the
specified field from the Table.
The result is a Tcl keyed list:
Note that most of the keys are similar to the FITS ASCII and Binary Table header
keywords, without the trailing "index."
However, there are some exceptions as to how they should be interpreted.
Clear information from the
specified field from the Table.
Any previous value is destroyed (character strings are set to null strings and
numeric values are set to zero).
The following member values can be cleared:
Set information about the
specified field from the Table.
Any previous value is destroyed.
The following members can have their values set:
Return the value of the
specified field from the
TBLCOL format Table.
If the requested field is an n-dimensional array, part or all of the
array may be returned.
If all indices are specified, then one array element (a scalar) is returned.
Otherwise, a Tcl list of values is returned.
Data stored indirectly through the ARRAY
(
Consider an example with a 4-dimensional array. The first index is the
Binary Table row, the remaining three indices (in row-major order) are
those from the field's TDIMn keyword (which are specified in
column-major order).
The example array dimensions from field
Returned values are decimal.
TDISP from a field's
TBLFLD is not
used to format the output of tblFldGet.
If the field is a character string (data type `A'), the actual
character string is returned, rather than a numeric representation of it.
The character string is not quoted with braces ({}).
Consider an example where a Table has 2 rows and the field has 1
character string.
A single character from the string is gotten with:
Values can also be returned from data in
heap.
Heap data is described through
TBLHEAPDSC structures
contained in an ARRAY.
This information is indexed in the manner as described above.
tblFldGet will return the heap data itself, not the
heap descriptors (TBLHEAPDSC) in the ARRAY data area.
Consider an example where each row in the field's ARRAY
contains 1 TBLHEAPDSC.
Specifying no indices will return all variable length arrays for all rows
in the field. Each variable length array is considered to be 1-dimensional:
When character strings are returned as part of an array of character strings
(object schema
STR),
they are quoted to allow Tcl lists to represent the array.
This behaviour brings out a problem.
Consider a character string that contains one or more quote character
(open or close brace ({})) that is returned as part of an array
of character strings (here, the field contains 2 character strings per row):
NOTE:
If tblFldGet were to quote the brace with a backslash
(\), behaviour would not be as expected. Consider:
Set the value or values of the
specified field in the
TBLCOL format Table.
If the requested field is an n-dimensional array, part or all of the
array may be set.
The starting array element to be set is determined from the indices.
For an n-dimensional array, if fewer than n indices are
provided, the missing (trailing) indices are treated as zeros.
ARRAYs which store data indirectly
(
NOTE:
Values cannot be set in heap.
From the starting array element, the values are sequentially
assigned to consecutive array elements.
This is possible as arrays are
laid out such that the
fastest varying indexed array elements are stored next to each other.
This does permit the list of values to span across "subarray"
boundaries.
Subarrays are based on the depth within the tree representing the
hierarchy of array pointers.
Consider a 3-dimensional integer array with dimensions (2, 3, 2) in field
The indices first array element and the last array element to be set are
returned. Again, consider the above 3-dimensional array:
NOTE:
Setting numeric values that are too large for the array data type
(such as 32768 for a signed 2 byte integer) results in an undetermined value
being assigned (-32768 is a possibility, but not guaranteed).
The outcome (resulting value and/or action such as an exception) is dependent
upon the machine and operating system.
Character strings (object schema
STR)
are handled somewhat differently than numeric or
LOGICAL data.
Although the array data elements are individual characters for STR,
tblFldSet works with complete null-terminated strings.
Tcl list elements, which the values parameter is comprised of,
are delimited by whitespace (blanks, tabs, etc.). Character strings containing
whitespace must be quoted.
Consider a list with 2 character strings:
Supplied indices still indicate at which character
assignments are to begin at.
But, successive characters in the values are not
necessarily assigned to successive array elements.
Consider a 3-dimensional STR array of dimensions (2, 2, 5).
The fastest varying index, 5 here, is the maximum length of any string
(which includes the one character expected for
guaranteed null-termination).
The setting of each string from values starts at the beginning of the
string within the array (the subarray referenced by the second fastest varying
index).
The characters `
tblFldSet's result,
the indices of the starting and ending array elements that were set,
reflects two points:
By default, if one of the character string values is too long,
no strings are assigned to the array and the command fails.
The -trunc option can be used to override this behaviour.
Strings which are too long are truncated as they're assigned to the array.
The truncated strings are still null-terminated properly.
No warnings are provided if any strings are truncated.
Setting character strings is troublesome if the value is to contain one or
more quote characters (open of close brace ({})).
With such characters, the result from
tblFldGet cannot be used directly
There is a further limitation on string values: namely, the first string
value may not begin with a dash (-):
Return a Tcl true (1) or false (0) value indicating whether the Table, if
written out as either an FITS ASCII or Binary Table extension, would comply
with the FITS Standard.
If the Table would not map to a legal FITS header and data unit (HDU),
tblCompliesWithFits will return with an error rather than a true/false
value.
The selection of the FITS extension type, through either the -ascii
or -binary options, is required.
It is important to note that tblCompliesWithFits tests the
potential for a TBLCOL Table to be rendered
FITS-compliant rather than actually being FITS-compliant.
This subtle distinction applies when writing Binary Tables whose
integer object schemas do not translate
directly to a FITS-compliant data type. In such cases, the Table writer
attempts to use the FITS community convention of setting
TSCALn and TZEROn to
flip the "signedness"
of the data type.
Heap Data
tblFldInfoGet
{COL 5} {TYPE ULONG} {DIM {2 3 4}} {NSTAR 0} {EMPTY 0} ...
The following keys are always returned:
The following keys might be returned, depending on whether they are being used:
The condition of some keys not being returned will need to be tested for,
either with catch or returning the keylget value to a
variable. Consider testing for TTYPE:
dervish> set fldInfo [tblFldInfoGet $binHand -col $tblFld]
{COL 5} {TYPE ULONG} {DIM {2 3 4}} {NSTAR 0} {EMPTY 0}
dervish> if {![catch {keylget fldInfo TTYPE}]} {TTYPE-code}
dervish> if {![keylget fldInfo TTYPE fldTTYPE]} {TTYPE-code}
tblFldInfoClr
tblFldInfoSet
tblFldGet
nStar
is non-zero) is
not returned.
$tblFld
are
(15,4,3,2)
.
A scalar is gotten with:
dervish> tblFldGet $binTbl -col $tblFld { 0 1 2 1 }
25890
A 1-dimensional array is retrieved by not specifying the fastest varying
(final) index:
dervish> tblFldGet $binTbl -col $tblFld { 0 1 2 }
3219 25890
A 2-dimensional array is retrieved by not specifying the last two indices:
dervish> tblFldGet $binTbl -col $tblFld { 0 1 }
{18 987} {2001 156987} {3219 25890}
All field values for all rows in the Table are returned if no indices are
are passed:
dervish> tblFldGet $binTbl -col $tblFld
{{18 987} {2001 156987} {3219 25890}} {...} ...
dervish> tblFldGet $binTbl -col $tblFld { 0 2 }
c
The complete string from one row is gotten with:
dervish> tblFldGet $binTbl -col $tblFld { 0 }
abcdef
Character strings from all rows are gotten with:
dervish> tblFldGet $binTbl -col $tblFld
{abcdef} {xyz}
dervish> tblFldGet $binTbl -col $tblFld
{35 987 2053} {} {-93 1045 28769 1534 100}
Specifying only the Table row index, namely to access the
TBLHEAPDSC will return the complete variable length array in
heap associated with that Table row:
dervish> tblFldGet $binTbl -col $tblFld { 0 }
35 987 2053
Finally, an individual variable length array element is returned by
specifying an index beyond the number of dimensions of the field's
ARRAY:
dervish> tblFldGet $binTbl -col $tblFld { 0 1 }
987
NOTE:
Because there may be some controversy as to what the FITS keyword
TDIMn really means for variable length data (it should be
noted that the FITS Binary Table Standard lists TDIMn
as a convention for multidimensional arrays, not as part of the Standard),
tblFldGet restricts ARRAYs of TBLHEAPDSCs
to be 1-dimensional.
This is in keeping with Dervish's maintenance of
FITS compliance
and Dervish Table consistency throught Dervish.
Limitations with Character Strings
dervish> tblFldGet $binTbl -col $tblFld { 0 1 }
{abc}def} {xyz}
Problems will occur when trying to access one of the array elements; consider:
dervish> set x [tblFldGet $binTbl -col $tblFld { 0 1 }]
{abc}def} {xyz}
dervish> lindex $x 0
Error: list element in braces followed by "def}" instead of space
dervish> set x [tblFldGet $binTbl -col $tblFld { 0 1 }]
{abc\}def} {xyz}
dervish> string index [lindex $x 0] 3
\
Notice that the backslash (\) is treated by Tcl as part of the
string rather than quoting the character following it.
tblFldSet
nStar
is non-zero)
may not have values assigned to array elements.
$tblFld
.
Array elements, starting at indices (1, 0, 1), can be set with
dervish> tblFldSet $binTbl -col $tblFld {1 0 1} {9 4 3 7}
{1 0 1} {1 2 0}
Pictorally, the following array elements are set:
level
-----
-1 ___.___
/ \
/ \
0 _0_ _1_ <------+-- slowest varying indices
/ \ / \ |
/ \ / \ | hierarchy of array
1 0---1---2 0---1---2 | pointers
/_\ /_\ /_\ /_\ /_\ /_\ |
0 1 0 1 0 1 0 1 0 1 0 1 <--+-- fastest varying indices
+-+-+-+-+-+-+-+-+-+-+-+-+ --.
2 | | | | | | | |9|4|3|7| | | data (array elements)
+-+-+-+-+-+-+-+-+-+-+-+-+ --'
corresponding to array elements indexed by (1, 0, 1), (1, 1, 0), (1, 1, 1), and
(1, 2, 0) respectively.
dervish> set range [tblFldSet $binTbl -col $tblFld 1 {8 21 5}]
{1 0 0} {1 1 0}
dervish> set elem(first) [lindex $range 0]
1 0 0
dervish> set elem(last) [lindex $range 1]
1 1 0
Setting Character Strings
dervish> tblFldSet $binTbl -col $tblFld 0 {{Spaces galore} hi}
It is inadequate to only escape the whitespace (with a backslash
(\)). Tcl strips out the escape character and tblFldSet
subsequently detects 3 character strings (using the above example).
dervish> tblFldSet $binTbl -col $tblFld {0 1 2} {a bc def}
{0 1 2} {1 1 3}
Pictorally, the following array elements (characters) are set:
_______._______
/ \
/ \
__0__ __1__ <--------- slowest varying indices
/ \ / \
/ \ / \
__0__ __1__ __0__ __1__ <---- string referencing indices
/ \ / \ / \ / \
/_______\ /_______\ /_______\ /_______\
0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 0 1 2 3 4 <-- fastest varying indices
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ (character referencing
| | | | | | | |a|0| |b|c|0| | |d|e|f|0| | indices)
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The trailing `0
's are the null-terminators.
Each string set in the array is terminated with a null character.
There is no means of setting one or more characters without them being
treated as null-terminated strings.
a
', `b
', `c
',
`d
', `e
', and `f
' from the strings
are not all assigned to adjacent array elements (as would be the case for
numeric or LOGICAL data).
In the above example, the first string, `a
', is assigned to the
character indexed by (0, 1, 2) (within the string starting at (0, 1, 0)).
The second string, `bc
', is assigned to the string
starting at the character indexed by (1, 0, 0).
The last string, `def
' is assigned to the string starting at the
character indexed by (1, 1, 0).
Limitations with Character Strings
dervish> set str [tblFldGet $binTbl -col $tblFld 0]
{abc}def} {xyz}
dervish> tblFldSet $binTbl -col $tblFld 0 "$str"
Error: list element in braces followed by "def}" instead of space
It is possible to set strings containing quotes by enclosing the overall
list of strings needs with Tcl's double quotes (").
These permit string substitutions at the Tcl level.
dervish> tblFldSet $binTbl -col $tblFld 0 "abc}def {xyz}"
This solution brings about its own set of conditions.
They apply to each string in the list that contains, in this case,
brace quotes (either { or }):
As mentioned, this solution applies to a list of strings that was originally
quoted with braces ({}). Similar solutions exist for lists
quoted with Tcl's other quotes.
dervish> tblFldSet $binTbl -col $tblFld 0 -strVal
ftclFullParse error: Too few parameters entered
Error: Parsing error
USAGE: tblFldSet <handle> ...
The solution is to make sure that the there are enough quotes
(braces ({})) around the string:
dervish> tblFldSet $binTbl -col $tblFld 0 {{-strVal}}
The outer braces are stripped, as they enclose the list of values; in this case,
the list contains only one value.
The inner braces surround the first string value.
These too are stripped, as each string value has any quoting stripped
(the values parameter is treated as a Tcl list of lists).
tblCompliesWithFits