setup primer

QUESTIONS

  • 1. What does the "setup" command do for me?
  • 2. The setup command doesn't work. What happened?
  • 3. How can I tell if setup works?
  • 4. What happens when I setup a package?
  • 5. What is available to be setup?
  • 6. How often do I need to setup dervish?
  • 7. How do I know what version is setup?
  • 8. How do I get different versions setup?
  • 9. Where does this all come from?
  • 10. How do ups and upd work to give me products?
  • 11. What else is there to read?
  • 12. I want to do everything. What do I set up, and in what order?
  • 1. What do I need to be able to install SDSS software on my local machine?
  • 2. How can I tell if I have proper upd and access?
  • 3. What kind of directory structure should I have?
  • 4. How do I use upd to install a product.
  • 5. Is there an easier way?
  • ANSWERS

    1. What does the "setup" command do for me? This is the FNAL standard way of defining software packages. If we only had to worry about one executable file, then we would simply copy it to a subdirectory defined in your PATH and you would be able to run the program directly. But things are more complicated. A software package in general contains: - executable(s) - documentation - source code - include files - initialization scripts - help - compiled libraries - examples and perhaps even more. Add to that, that we support: - versions - flavors of machines and you can see that software distribution involves a lot more than just getting an executable image. When you say "setup dervish" you define the variable $DERVISH_DIR which is the root for all of the subdirectories you need, given the latest official version and the machine you are running on. Once that step is finished, you can refer to the files using the same root name, even after the versions or flavors change. 2. The setup command doesn't work. What happened? If you are just starting, and try this command: % setup tex You might get an error, such as /usr/bin/setup: Permission denied. Or, it might complain that the command "setup" is not defined. The trick is that there needs to be initialization commands run that define what setup does. They are (at Fermilab): source /usr/local/etc/fermi.login source /usr/local/etc/fermi.cshrc Now, this should be done for you automatically when you login, for example by having these commands in the /etc/cshrc file in the system. If this does not happen automatically for you, talk to the local expert to see where the fermi.login and fermi.cshrc files are located on your machine. The bare essentials that these scripts do to set up ups and upd are, for example, at Princeton: #!/bin/csh # setup ups environment setenv PRODUCTS /u/streets/ups_database setenv UPS_DIR /u/streets/products/ups/v2_5 source $UPS_DIR/ups/ups_init.csh setup funkern setup ups setenv MANPATH "/u/streets/man:$MANPATH" 3. How can I tell if setup works? Try the following: % setup ups If you get a prompt back with no complaining, then it worked. 4. What happens when I setup a package? At the very least, an environment variable is defined for that package: % setup ups % printenv | grep UPS UPS_DIR=/usr/products/ups/v2_1 Other packages are more involved, and define several environment variables: % setup tex % printenv | grep TEX TEXTOOLS_DIR=/usr/products/textools/v1_0 TEX_DIR=/usr/products/tex/v3_1 TEXFONTS=.:/usr/products/tex/v3_1/lib/tex/fonts/tfm TEXFORMATS=/usr/products/tex/v3_1/lib/tex/formats TEXINPUTS=.:/usr/products/tex/v3_1/lib/tex/inputs TEXPOOL=/usr/products/tex/v3_1/lib/tex It also adds something to PATH so you can get at executables: % echo $PATH /usr/products/tex/v3_1/bin:/usr/sbin:/usr/bsd:/usr/bin:/bin:/usr/bin/X11: /usr/local/bin:/sky/stoughto/bin:. 5. What is available to be setup? There is a ups command that lists the current version of everything that is available on the machine: % ups list If you are interested in older versions: % ups list -a If this list is too long and you are interested in just one package: % ups list -a pgplot gives the list for just the pgplot package. 6. How often do I need to setup dervish? One time in each window or login session. You can say: % setup dervish And then as many times as you want: % dervish dervish> quit % dir $DERVISH_DIR or whatever. That product stays set up as long as that window is logged in. You have to setup the product in EACH window that you open. 7. How do I know what version is setup? There is usually a clue in the value of $*_DIR. For example, we have version 3.1 of TEX and version 2.1 of UPS in the examples above. Wise system people declare sets of "current" versions that are supposed to be pretty safe to use. You get the current version by default. There is also a "development" version that could be newer, and better. 8. How do I get different versions setup? % setup pgplot development gets you the development version; % setup pgplot v4_9ef0 gets you a particular version you are interested in. Unless you are an expert, stick with the default (current) version. 9. Where does this all come from? There is another product called upd that retrieves packages from servers at FNAL. In general, you need to sign up to be able to have access to these machines. There should be at least one local expert who knows how to do this. 10. How do ups and upd work to give me products? upd stands for "user product distribution". ups stands for "user product support". A software package such as tcl or pgplot is called a product. A product can be anything, but typically it is a collection of interrelated source code (such as a plotting package), libraries, binaries, and supporting documentation. Products come in several versions, hopefully each one new and improved over the older versions. A given version of a product should be stable - everyone in our collaboration hould have identical copies. The exception is a development version, which is continuously changing. upd and ups are both products themselves. The functions that they serve are to: a) copy new versions of products from a central computer to your own computer; b) maintain a database of what products you have stored on your computer and where they are (i.e. the directory location); c) provide a simple mechanism to define any logical names (environment variables, etc.) that you need in order to use a product; d) provide a common environment independent of the type of machine that you are using. Among other things, that means that when we write the manual on "How to Do This and That", we don't need to have many comments like "tailor this command for your own system". upd performs functions a) and b) above. ups performs part of function b) and function c). The entire suite accomplishes d). ups and upd need to be installed on your system before they can be used. Fermilab will likely do this for one system at each institution on the project. As of this writing only Princeton, Fermilab, and Japan have them installed. One user at each institution should be knowledgeable enough to use upd/ups to copy/install new versions of products. If you are a random user, you should not need to know the details of how upd/ups work. The one thing you should know about are the setup and unsetup commands. If you are lucky, your system administrator will make sure that these are defined every time that you log in. 11. What else is there to read? The manual page for setup: % man setup % man ups 12. I want to do everything. What do I set up, and in what order? % setup www % setup dss % setup dervish ============================================================================= The following information is for installing software via upd. Most people do NOT need to worry about this. The local contact person should have done this for you already. If YOU are the local contact, read on. 1. What do I need to be able to install SDSS software on my local machine? You need to be able to run the upd program on at least one of your machines. You also need to have permission granted on two machines at Fermilab -- fndaub and fnsg01. Finally, you need to have access to disk space on your machine -- hundreds of Mbytes. 2. How can I tell if I have proper upd and access? Try the following: pisces% source ~products/setenv_ups.csh (On pisces, you must do this to have "setup" work) pisces% setup upd (If it does not complain about "product upd not found" then you have it. To run upd: pisces% upd Welcome to the Menu interface for upd - choose one of the following options: 1 upd list: list available products 2 upd copy: copy a product from a remote node 3 upd unwind: unwind product into local directory 4 upd tarcont: list the table of contents of the tar file 5 upd extract: extract one or more files from tar file 6 upd get req: get the build and use requirements 7 ups declare: declare the product to UPS 8 ups tailor: execute products ups/tailor file 9 fork a shell: spawns a login shell q quit Enter choice : (The whole thing is menu based. First, exercise the "list" function. Enter a 1. This gives you the following menu:) UPD LIST: * implies must be set *1 [] remote node name 2 [] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : (Set the "remote node name" by entering a 1. Then the menu looks like:) Enter choice : 1 Enter remote node name []: fndaub.fnal.gov UPD LIST: * implies must be set *1 [fndaub.fnal.gov] remote node name 2 [] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : (You can set more choices in a similar way. Set product name: ) Enter choice : 2 Enter product name (hit return for all products): dervish UPD LIST: * implies must be set *1 [fndaub.fnal.gov] remote node name 2 [dervish] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : (Finally, execute the command with the e option) UPD sending request to remote node... Distributable Versions of dervish development --> devel Flavor: IRIX current --> v3_2 Flavor: SunOS no_chain: v3_1 Flavor: SunOS UPD LIST: * implies must be set *1 [fndaub.fnal.gov] remote node name 2 [dervish] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : Since this listing worked, we know that we are allowed upd access to fndaub. 3. What kind of directory structure should I have? The general path name should be under a "products" account. It will work from any root, but we suggest a separate account for your local machine to copy in these products. Let us assume that the root will be ~products. Under that branch, you need two subdirectories: one for IRIX and the other for SunOS. Even if you have only one flavor at your place, please set up these two trees so that all sites will be similar. Assume now that you want to install a product called "newprod" for the SunOS. In ~products/SunOS, make a directory called newprod. \ In ~products/SunOS/newprod make a directory for the version, for example v1_0. 4. How do I use upd to install a product. - make sure it is available, from fnsg01 or fndaub (upd list) - make a directory for this version of the product (unix commands) - copy the product over (upd copy) - unpack the tar file (upd unwind) - declare the product (upd declare) - test (in unix, setup the product; see if it is there) The following is a dump of the screen when I installed the product tk for the first time on pisces: pisces% cd /home1/products pisces% cd SunOS /home1/products/SunOS pisces% dir diff/ jpg1/ photo_data/ dervish/ fsao/ libfits/ rcs/ upd/ ftcl/ murmur/ rcvs/ ups/ fulib/ perl/ sdssrcvs/ www/ funkern/ pgplot/ setenv_ups.csh gccfix/ photo/ setenv_ups.csh~ pisces% cd /home1/products pisces% cd SunOS /home1/products/SunOS pisces% dir diff/ jpg1/ photo_data/ dervish/ fsao/ libfits/ rcs/ upd/ ftcl/ murmur/ rcvs/ ups/ fulib/ perl/ sdssrcvs/ www/ funkern/ pgplot/ setenv_ups.csh gccfix/ photo/ setenv_ups.csh~ pisces% mkdir tk pisces% cd tk /home1/products/SunOS/tk pisces% upd Welcome to the Menu interface for upd - choose one of the following options: 1 upd list: list available products 2 upd copy: copy a product from a remote node 3 upd unwind: unwind product into local directory 4 upd tarcont: list the table of contents of the tar file 5 upd extract: extract one or more files from tar file 6 upd get req: get the build and use requirements 7 ups declare: declare the product to UPS 8 ups tailor: execute products ups/tailor file 9 fork a shell: spawns a login shell q quit Enter choice : 1 UPD LIST: * implies must be set *1 [] remote node name 2 [] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : 1 Enter remote node name []: fndaub.fnal.gov UPD LIST: * implies must be set *1 [fndaub.fnal.gov] remote node name 2 [] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : 2 Enter product name (hit return for all products): tk UPD LIST: * implies must be set *1 [fndaub.fnal.gov] remote node name 2 [tk] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : e UPD sending request to remote node... Distributable Versions of tk current --> x0 Flavor: IRIX current --> x0 Flavor: SunOS UPD LIST: * implies must be set *1 [fndaub.fnal.gov] remote node name 2 [tk] product name 3 [] instance 4 [] flavor 5 [] remote database e execute upd list q quit this menu Enter choice : q Welcome to the Menu interface for upd - choose one of the following options: 1 upd list: list available products 2 upd copy: copy a product from a remote node 3 upd unwind: unwind product into local directory 4 upd tarcont: list the table of contents of the tar file 5 upd extract: extract one or more files from tar file 6 upd get req: get the build and use requirements 7 ups declare: declare the product to UPS 8 ups tailor: execute products ups/tailor file 9 fork a shell: spawns a login shell q quit Enter choice : 2 UPD COPY: * implies must be set *1 [fndaub.fnal.gov] remote node name *2 [tk] product name *3 [] instance 4 [] flavor *5 [] temporary file for tar 6 [] remote database e execute upd copy q quit this menu Enter choice : 3 Enter instance []: x0 UPD COPY: * implies must be set *1 [fndaub.fnal.gov] remote node name *2 [tk] product name *3 [x0] instance 4 [] flavor *5 [] temporary file for tar 6 [] remote database e execute upd copy q quit this menu Enter choice : 5 Enter name for tar file []: /home1/products/SunOS/tk/x0.tar ****NOTE THAT I USED AN ABSOLUTE PATH NAME. ALWAYS THE SAFEST WAY TO DO THIS UPD COPY: * implies must be set *1 [fndaub.fnal.gov] remote node name *2 [tk] product name *3 [x0] instance 4 [] flavor *5 [/home1/products/SunOS/tk/x0.tar] temporary file for tar 6 [] remote database e execute upd copy q quit this menu Enter choice : e Enquiring tar file name from remote node ... Copying tar directory ... ************** THIS WILL TAKE A WHILE..... UPD COPY SUCCEEDED: continue with unwind Welcome to the Menu interface for upd - choose one of the following options: 1 upd list: list available products 2 upd copy: copy a product from a remote node 3 upd unwind: unwind product into local directory 4 upd tarcont: list the table of contents of the tar file 5 upd extract: extract one or more files from tar file 6 upd get req: get the build and use requirements 7 ups declare: declare the product to UPS 8 ups tailor: execute products ups/tailor file 9 fork a shell: spawns a login shell q quit Enter choice : 3 UPD UNWIND: * implies must be set *1 [/home1/products/SunOS/tk/x0.tar] tar file name *2 [] home directory for product 3 [] key e execute upd unwind q quit this menu Enter choice : 2 Enter home directory []: /home1/products/SunOS/tk/x0 UPD UNWIND: * implies must be set *1 [/home1/products/SunOS/tk/x0.tar] tar file name *2 [/home1/products/SunOS/tk/x0] home directory for product 3 [] key e execute upd unwind q quit this menu Enter choice : 3 . . . long listing of the file names as they are unwound . . . Enter choice : 7 UPS DECLARE: * implies must be set *1 [tk] product name *2 [x0] instance 3 [] flavor *4 [/home1/products/SunOS/tk/x0] home directory 5 [] UPS sub directory 6 [] database 7 [< ftcl v1_6 SunOS] use requirements 8 [] build requirements 9 [-c] declare current 10 [] copy files toInfo e execute ups declare q quit this menu Enter choice : 3 Enter flavor (hit return for default): SunOS UPS DECLARE: * implies must be set *1 [tk] product name *2 [x0] instance 3 [SunOS] flavor *4 [/home1/products/SunOS/tk/x0] home directory 5 [] UPS sub directory 6 [] database 7 [< ftcl v1_4 SunOS] use requirements 8 [] build requirements 9 [-c] declare current 10 [] copy files toInfo e execute ups declare q quit this menu *** Make SURE you SET THE FLAVOR!!! Filling the product structure Creating a new instance Got the use requirements check for circular dependencies check for circular dependencies Reading product ftcl from the database Added the new instance Creating current chain Special case to make current cannot execute /home1/products/SunOS/tk/x0/ups/current, no such file writing the product structure Successfuly declared the product UPS DECLARE SUCCEEDED: try UPS TAILOR or another product or quit Welcome to the Menu interface for upd - choose one of the following options: 1 upd list: list available products 2 upd copy: copy a product from a remote node 3 upd unwind: unwind product into local directory 4 upd tarcont: list the table of contents of the tar file 5 upd extract: extract one or more files from tar file 6 upd get req: get the build and use requirements 7 ups declare: declare the product to UPS 8 ups tailor: execute products ups/tailor file 9 fork a shell: spawns a login shell q quit Enter choice : q pisces% setup tk pisces% cd $TK_DIR /home1/products/SunOS/tk/x0 pisces% 3. How do I know what the build requirements are for a product? This information is transmitted with the ups copy command. If you want to find out directly, use the ups list -u command on a machine that is already set up, for example, fncase: fncase % ups list -b dervish dervish v3_2: current version Home: /usr/products/SunOS/dervish/v3_2 BUILD REQUIREMENTS ORDER: < PRODUCT: tk VERSION: current FLAVOR: SunOS ORDER: < PRODUCT: fsao VERSION: v2_0 FLAVOR: SunOS ORDER: < PRODUCT: gcc VERSION: v2_4_1 FLAVOR: SunOS ORDER: < PRODUCT: gccfix VERSION: current FLAVOR: SunOS ORDER: < PRODUCT: ftcl VERSION: v1_6 FLAVOR: SunOS ORDER: < PRODUCT: libfits VERSION: v1_1 FLAVOR: SunOS ORDER: < PRODUCT: pgplot VERSION: v4_9ef3 FLAVOR: SunOS ORDER: < PRODUCT: tex VERSION: v3_1 FLAVOR: SunOS fncase % Also, the release notes for the product might also have this information listed. 4. What node do I get the products from? Either fndaub or fnsg01. For the latest release of DERVISH, for example: product version flavor $*_DIR upd node ------- ------- ------ ------ --- ---- tk current IRIX /usr/products/IRIX/tk/x0 fndaub www current IRIX /usr/products/IRIX/www/v1_4 fnsg01 fsao v2_0 IRIX /usr/products/IRIX/fsao/v2_0 fndaub tex v3.1 IRIX /usr/products/IRIX/tex/v3.1 fnsg01 ftcl v1_6 IRIX /usr/products/IRIX/ftcl/v1_6 fnsg01 libfits v1_1 IRIX /usr/products/IRIX/libfits/v1_1 fndaub pgplot v4_9ef3 IRIX /usr/products/IRIX/pgplot/v4_9ef3 fndaub 5. Is there an easier way? Yup. Use upr. It "automatically" does many of these steps. Contact Vijay Gurbani to get your computer set to use upr.