TCL primer

QUESTIONS

  • 1. What is TCL?
  • 2. How much do I have to learn about TCL?
  • 3. What supporting documentation is there?
  • 4. OK, where should I start?
  • 5. What do all of the special characters mean?
  • 6. How are lists handled?
  • 7. What are TCL procedures?
  • 8. How would I build a command to read in a FITS file?
  • 9. If I forget what is in a region, what do I do?
  • 10. How do I repeat the same operation on a bunch of regions or files?
  • 11. How can I get a list of files on disk?
  • 12. I like these commands. Do I have to type them in every time?
  • 13. What is included in ftcl?
  • 14. What is included in tclx?
  • 15. What else do I need to know?
  • ANSWERS

    1. What is TCL? TCL is basically a "C and Lisp" like command line interpreter (CLI). It serves much the same function as "cl" in IRAF or the CLI that you may have used in other image processing application programs. For users of Mirella, TCL provides the functionality of the Forth interpreter. TCL is used typically for interactive type programs where you type a command and something happens. TCL is also a programming language. In fact, it has similar but much more powerful capabilities as the shell environment in Unix or the DCL environment in VMS: in the simplest cases you just type commands and something happens, but in more complex cases you can create loops, set variables, do simple arithmetic, etc. The distinction of TCL is that it and all the functions that it makes happen are linked into a single executable program. Because TCL is a programming language, you can actually write applications entirely in TCL itself; however, TCL is hardly a substitute for writing in C or Fortran for more CPU intensive applications. A TCL application (such as the photometric pipeline) thus consists of two parts: the code that implements TCL itself and the application code that you or someone else writes. There are a number of utility routines to aid in integrating your code into the TCL environment. Typically, one writes the heavy duty application code in C and then creates TCL commands to call those functions. The TCL package itself (which is common to all TCL applications) is a free software package from John Oosterhout of Berkeley. 2. How much do I have to learn about TCL? There are two aspects to TCL that you will want to learn. The first is TCL itself as a programming language. If you are going to be operating the online control system for SDSS, this is all you really need. The second is how to add your application code into the TCL environment so you can call your code via a TCL command. 3. What supporting documentation is there? At present things are a bit scattered. All existing documentation has been distributed with the photometric pipeline. To find it, do the following. If the products are not already setup, type: % setup ftcl % setup dervish Then the documenation can be found as follows. $FTCL_DIR/doc/tclUsenix90.ps describes the TCL programming language and is the place to start, along with tclUsenix91.ps $FTCL_DIR/doc/*.man The remaining files describe TCL C routines that are used to interface your C code into the TCL environment. Each .txt file has a corresponding .man page with duplicate information. $FTCL_DIR/doc/pn464.ps Fermilab cannot leave things untouched. A few extensions have been made to TCL that are described here. The most useful are command line editing via the arrow keys that VMS users know and love. $FTCL_DIR/extended/man/tclx.man Well, TCL comes in two parts: standard and extended. The extended features were added by Oosterhout after the original was created. For your purposes, there is no need to distinguish between the two, except that the documentation of the extended features is here. This documentation covers the extensions to the TCL language itself. $FTCL_DIR/extended/man/*.man All the other .man files are documentation for the additional interface routines for the C programmer. All the .man pages (including TclX) should be converted to .txt files to make printing easier. Note that the functions that one invokes by TCL commands can often be called directly as C functions. There are usually differences in detail as to how parameters are passed. 4. OK, where should I start? The first thing you want to learn is TCL the programming language. So you should print out the first document above. Now TCL the programming language is part of every TCL application that we create. A bare-bones version is called "dervish_template" and is the starting point for building new pipelines. It does not really do anything, but it is a good place to start learning TCL. % setup dervish (If you have not already done so) % dervish Executing commands in /usr/products/IRIX/dervish/v2_1/etc/dervishStartup.tcl: dervish> (Note that a startup script is run when you begin. This is where the command "dervishPlot" is defined, as an example of using scripts to define new commands.) It is probably easiest to learn TCL on your own simply by reading the TCL manual and trying out lots of examples. Here, however, are some pointers that may clarify some peculiar aspects of the language. The basic format of commands as you give them to TCL is simple: dervish> command parameter1 parameter2 .... with spaces separating the various items. A simple example of a TCL command is dervish> set par 22 22 This creates a new variable par (if it does not already exist) and gives it the value 22. All variables and their contents are treated as strings, so 22 is actually stored as the ascii characters 22. Where necessary (in evaluating expressions), TCL will convert ascii numbers to binary, but any binary results are reconverted back to ASCII strings. [This makes TCL a not very good programming language for numerical manipulation. C'est la vie]. TCL commands return two things: a result and a condition code. - The condition code can usually be ignored (it is either OK or ERROR). - The result serves the same purpose as results returned by any C function and is the primary mechanism to pass information between TCL commands. The result is again always in the form of an ascii string. The set command returns the value that a particular parameter was set to. Thus, in the above example, the string 22 is returned. In this case it is just echoed to the terminal. Note that if you type dervish> set par 22 with no second parameter, it will return the current value of par. 5. What do all of the special characters mean? TCL has several reserved characters that perform useful operations. Some of them are: $var Extract the value of var {} Grouping - do not expand $ inside here " " Grouping - DO expand $ inside here [ ] Execute the enclosed thing as a TCL command; expand $ inside These are used as in the following examples: Dollar signs: ------ ------ dervish> set a apple apple dervish> set b $a apple The $ causes the value of a to be extracted, so the 2nd statement translates to dervish> set b apple apple Braces: ------- dervish> set a {apple orange} apple orange Braces cause their contents to be treated as a single item with no translation or special treatment of anything inside (similar to the way TeX uses them). The braces are stripped off when then contents are assigned to a. Without braces, this statement would produce an error. Typing dervish> set b $a apple orange translates to dervish> set b {apple orange} apple orange so braces are implicitly added to a translated item if needed (but then removed immediately once the item is used). Quotes: ------- dervish> set b "$a banana" apple orange banana translates to: dervish> set b {apple orange banana} apple orange banana Thus, quotes act like braces except that translation of variables occurs. (and evaluation of things in brackets - see next item). Brackets: --------- dervish> set b [set a] apple orange Anything inside brackets is treated as a TCL command. The whole thing is replaced by whatever result is returned by the execution of the tcl command. Thus the above statement translates to: dervish> set b {apple orange} apple orange once again. 6. How are lists handled? Lists are the most useful composite data type in TCL. Some functions want to return more than one value as their results. The TCL mechanism for doing so is by way of a list (FORTH programmers: lists provide the same functionality as stacks for this purpose). A list is a set of items in a character string separated by blanks. TCL has several commands for manipulating lists. In the example dervish> set a {apple orange} apple orange the item in braces is a list. By enclosing it in braces, the list is treated as a single item as far as the set command is concerned. The command llength counts the number of items in a list. Thus, dervish> llength $a 2 translates to dervish> llength {apple orange} 2 and the value 2 is returned. Lists can be nested. For example, {3 2} {3 3} {3 4} is a list of lists. 7. What are TCL procedures? These are the TCL version of subroutines. The format is described in the TCL manual. Briefly, procedures are defined by the following statement: dervish> proc procname {par1 par2 ...} {body of TCL statements} Arguments are passed to the parameters par1 par2 ... and are available in the body of the procedure. Arguments are passed by value - thus, you cannot reset the value of par1 and have it returned without extra work. Procedure bodies are checked for syntax but are otherwise not interpreted when the procedure is defined. If a procedure calls another procedure, then the second procedure is searched for only when the first procedure is actually executed. This behavior is similar to the way TeX works in executing macro definitions, but it is different from the way FORTH works. You can redefine existing procedures; the new version replaces the old. This behavior makes life easy if you are developing and debugging new TCL procedures. 8. How would I build a command to read in a FITS file? The commands you want to hook together are regNew and regReadAsFits. You will find out about these commands in more detail in the regions primer. For now, remember that: regNew creates a place for a new region in memory regReadAsFits reads a fits file from disk and puts it in the region The following lines create a region and reads in a fits file to fill it: dervish> regNew h0 dervish> regReadAsFits h0 $INT_DIR/fits/m51.fits h0 This can be accomplished one line: dervish> regReadAsFits [regNew] $INT_DIR/fits/m51.fits h1 where the [regNew] command is executed and its result put in its place. Notice that we now have TWO copies of m51.fits in memory; one has the handle h0 and the other has the handle h1. We define a new command (procedure) with the proc command: dervish> proc rfits {fname} {regReadAsFits [regNew] $fname} The name of the procedure is rfits; the argument fname is used in the command defined between the {}'s. dervish> rfits $INT_DIR/fits/m51.fits h2 dervish> rfits $INT_DIR/fits/good.fit h3 9. If I forget what is in a region, what do I do? One command is regInfoGet with the region name: dervish> regInfoGet h0 {nrow 512 } {ncol 512 } {type S16 } {name h0 } {row0 0 } {col0 0 } {modCntr 0 } {isSubReg 0 } {isPhysical 0 } {pxAreContiguous 1 } {hasHeader 1 } {headerModCnt r 1 } {nSubReg 0 } This returns a list of parameters and their values, in a tcl list. If you suspect that there is another command that will give the information in a more useful form, try using help: dervish> help reg (This gives you a long list of all commands that dervish knows that have "reg" in the name.) The following procedure is defined in the dervishStartup.tcl file: dervish> regListNames h0 h1 h2 h3 (This gives you what you will want to use to operate on a list of regions.) 10. How do I repeat the same operation on a bunch of regions or files? Use the foreach command. It has the syntax dervish> foreach variable_name list body So you make a list with some command. Variable_name takes on the value of each member of the list (one at a time) and executes the body. For example dervish> foreach regname [regListNames] {echo Here's a region name: $regname} Here's a region name: h0 Here's a region name: h1 Here's a region name: h2 Here's a region name: h3 This is pretty useless, but if you want to clear out all your memory and regions, then do this: dervish> foreach regname [regListNames] {regDel $regname} dervish> regListNames Error: can't read "l2": no such variable So now we have a clean slate. In fact, we should make this a command: dervish> proc clrall {} {foreach rn [regListNames] {regDel $rn}} 11. How can I get a list of files on disk? The exec command passes strings back out to UNIX to be processed. So the following command will give you a directory of files in the current working directory: dervish> exec ls In this example, the fits files are located one directory above: modify the command to point to where they are on your machine. How about a command that lists the names of all files that have the string ".fit" in them? dervish> exec ls $INT_DIR/fits | grep .fit dithered.fit good.fit m51.fits m51_c1.fits m51_c2.fits m51_c3.fits (We use the grep command (get regular expression and print) instead of *.fit because "*" does not work.) We should make this a command which takes the subdirectory name as a parameter: dervish> proc lfits {dir} {exec ls $dir | grep .fit} dervish> lfits Error: no value given for parameter "dir" to "lfits" (Notice that the error message is not completely useless, since we used a good name for the parameter: dir for directory. If we define lots of commands with parameter names p1, p2, p3, and so on, these error messages will not be as useful.) dervish> lfits $INT_DIR/fits dithered.fit good.fit m51.fits m51_c1.fits m51_c2.fits m51_c3.fits We want to turn this into a list of file names. Use the split command: dervish> split [lfits $INT_DIR/fits] dithered.fit good.fit m51.fits m51_c1.fits m51_c2.fits m51_c3.fits Now we can use this in our command to read in file names. dervish> foreach fname [split [lfits $INT_DIR/fits] ] \ >> {echo $fname...; rfits $INT_DIR/fits/$fname} dithered.fit... good.fit... m51.fits... m51_c1.fits... m51_c2.fits... m51_c3.fits... This might make a useful command called rallfits. dervish> proc rallfits {dir} \ >> {foreach fname [split [lfits $dir ] ] {echo $fname...; rfits $dir/$fname}} Before we execute this reading again, lets clear out the memory a bit with the command we invented before: dervish> clrall dervish> rallfits $INT_DIR/fits dithered.fit... good.fit... m51.fits... m51_c1.fits... m51_c2.fits... m51_c3.fits... And if you want a list of USEFUL information about what you have read in, use the regionGetLine command to see what is in the header as the OBJECT name. If this is a different name for each region, then you will be able to identify the regions. dervish> foreach regname [regListNames] \ >> {echo $regname;\ >> echo [regHdrGetLine $regname OBJECT]} h0 OBJECT = '0100;1 ' h1 OBJECT = '0100;1 ' h2 OBJECT = 'm51 B 600s' / h3 OBJECT = 'm51 B 600s' / h4 OBJECT = 'm51 B 600s' / h5 OBJECT = 'm51 B 600s' / (Here we used a backslash to continue typing the same command on a new line, since the commands are getting long.) Which becomes the command called status with: dervish> proc status {} \ >> {foreach regname [regListNames] \ >> {echo $regname; echo [regionGetLine $regname OBJECT]} } 12. I like these commands. Do I have to type them in every time? No. You can put them all in a file. In this area, there is a file called commands.tcl which defines these commands. To execute it: dervish> source $INT_DIR/tcl/commands.tcl This is what the file looks like: # These are some illustrative and perhaps even useful tcl commands: echo Reading in sample tcl commands: proc rfits {fname} \ {regReadAsFits [regNew] $fname } proc lfits {dir} \ {exec ls $dir | grep .fit} proc rallfits {dir} \ {foreach fname [split [lfits $dir ] ] \ {echo $fname --- ; rfits $dir/$fname}} proc clrall {} \ {foreach rn [regListNames] \ {regDel $rn}} proc status {} \ {foreach regname [regListNames] \ { echo $regname; echo [regHdrGetLine $regname OBJECT]}} 13. What is included in ftcl? The Fermilab extensions to tcl are described in PN464: FTCL-TCL at Fermilab. The new features are: - command line editing (uparrow just like VMS!) - argument list parsing - help - Unix style command aliasing. 14. What is included in tclx? The extended tcl library included the following, stolen from the command: % man tclx o General Commands o Unix Access Commands o File I/O Commands o File Scanning Commands o Math Commands o List Maninipulation Commands o Keyed Lists o String and Character Manipulation Commands o XPG/3 Message Catalog Commands o Tcl Shell and Development Environment o Help Facility o Tcl Shell Initialization Sequence o Tcl Initialization File o Tcl Shell Variables o Tcl Shell Procedures o Tcl Command Autoloading o Tcl Package Libraries o Tcl Package Library Procedures 15. What else do I need to know? This material presents the basic elements of TCL. There are several major statement types and functions available that have not been covered - loops and if statements, for example. However, with this introduction, the remainder of the TCL manual can be read to provide the necessary information. You are now ready to read the tcl manual page: % man tcl overview of tool command language facilities % man library standard library of tcl procedures. % man tclx extended command set for tcl 6.2. There are further examples of tcl scripts and verbs in the following files.