leda Installing and maintaining a mirror site 
| HyperLeda home | Documentation | Search by name | Search near a position | Define a sample | SQL Search |

This document describes the procedure to install and maintain an HyperLeda mirror or the Pleinpot software. It corresponds to version 8.11 (May 2007).
All the software and data are freely available for download (GPL license), and the installation, including the WEB front-end, can easily be done on any Unix/Linux machine.
One of the goal of the development team is to provide a complete package with a minimum required platform, easy to install and easy to maintain. The present version is close to these goals.
Following the instructions on this page allows to install the databases and tools with a shell interface and/or http interface. Free support is provided for installation and operation.

Hardware/Software requirement

  1. Disk space
    For the standard installation, including the database and the Hypercat FITS archive the total disk space requirement will not exceed 20 Gb.

    The disk storage is divided in two parts:

    1. Main tree (software and catalogues)
      In total, including the hypercat FITS archive the volume is presently 15 Gb (The size of the HFA is 4.8 Gb, the catalogues of the database occupy 9 Gb and the executable programs 320 Mb The sources occupy 125 Mb, documentation libraries and tools to maintain the package are 100 Mb. About 300 Mb more are needed during installation.
    2. The WEB interface and documentation.
      The volume is 18 Mb
    Note that these volumes are just rough indications, they evolve (grow) with time and may vary a little from one site to the other.
  2. Shells and Compilers
  3. Libraries
  4. http server (required only if you install the web interface)
    The local http server must enable the execution of cgi scripts on the directory where HyperLeda is installed. These scripts are recognized from their extension: .cgi
    Important information about the configuration are reported in the file .htaccess that will be installed on the web root-directory.

First installation

(the present procedure has been updated on 2007/05/31)

The installation of Pleinpot should be done by a normal non-priviledged user that is refereed in the following as the Pleinpot super-user. Using 'root' superuser may be dangerous.

This installation of Pleinpot (and HyperLeda) follows a classical procedure: get the archive file, configure and make. The installation should be easy, but I encourage you to be extremely careful (files are created, so be sure that you now where they will go...).

The files created when installing Pleinpot and HyperLeda will be attached from the following three roots (details will be given later):

  1. PL_ROOT: The main top directory where the sources are located. The executable files, the libraries and the catalogues will be intalled from this root.
  2. CGIDIR: This is the directory where the web interface (HTML files, CGI script ...) will be installed (if you wish). This directory must pre-exist to the installation (create it if necessary) and must be reserved to HyperLeda (installation procedure erases other files!).
  3. PL_HTML: The directory where the HTML documentation of Pleinpot will be installed. This directory must pre-exist to the installation (create it if necessary) and must be reserved to Pleinpot (installation procedure erases other files!)

Now, how to proceed...

  1. Create a directory hosting the root of pleinpot (usually, it is called "pleinpot"), and move to this new directory.
  2. Retrieve to this directory the file: http://cral.univ-lyon1.fr/hldfc/pleinpot_ini.tar.gz
  3. Install the files from this archive file as: 
                     gunzip -c pleinpot_ini.tar.gz | tar -xf -
                 rm pleinpot_ini.tar.gz
    This file contains the minimum needed to configure the installation (0.3Mb). In particular, in contains the wget utility which will be installed if it is not available on the current host. The rest of the software will be downloaded (with wget) by the installation procedure.
  4. Depending if you want to install the WEB interface for HyperLeda and the Pleinpot documentation, you must choose the proper configure options.
  5. If you have Postgres available on your machine and a database server running, you may want to use this server for Pleinpot. Please read instructions for using a provided database server.
  6. Execute the configuration script that will produce the file "local_definitions.mk" containing the local dependencies to be used by the makefiles.
    The configuration script has been generated with autoconf, but some details of your local environment may be miss-interpreted. It is advised to check carefully the screen output of "configure". If you have problems you can re-run "configure" with other options or after setting (unsetting) some environment variables. Finally only if you are desperate you may edit the file "local_definitions.mk". I encourage you to report the problems you had, so I can improve the configuration script. (mail to leda@univ-lyon1.fr). 
              cd ${PL_ROOT}
          unix/configure [with options choosed above]
    Note that some of the variables produced by configure may (must) be altered by setting some environment variables before executing the script. Some command line options may also be used to customize the installation.

    Environment variables that affect configure behaviour

    Some other configure options
    There are other options, type unix/configure --help for the whole list. Several of them concern the database installation and operations. If your site is protected behind a fire-wall for http requests, you may need to configure with a proxy server. Some other options are:
  7. The rest of the installation depends on the actual elements you want to install. We propose three major installations: To install one of this distribution, type or
    These commands may take a long time. They are downloading files from the central Pleinpot archive and therefore they depends on the network load and bandwidth. The database installation may also be long, depending on the CPU you are using (between 1/2 and a couple of hours).
    If something goes wrong, contact Philippe Prugniel.
    It is possible to arrange variants of these installation using other targets described in the main makefile.
    Any of these installations create the HyperLeda WEB front-end and the html documentation for Pleinpot if the proper configuration switch where used with the command unix/configure [options].
  8. Test the installation
  9. Put the logo of your site
    Copy a gif file with the logo of your site to the ${HYP} directory and give it the name site.gif. For example do: If you do not do this, by default the logo will be the Hubble tune-fork icone.

Notes

  1. Most common installation problems:
    (1) Interrupted downloading. The first file installed by Pleinpot contains the minimum elements needed to continue the installation. The rest of the software and data are fetched from the network (but see the configuration options: it is possible to download from a cdrom). It may happen that the download is stopped due to network instability (and the message is unfortunately not always clear). The solution is to restart the installation (it will probably continue from the points it has been stopped). If the problem persists, send us a note.
    (2) If your are behind a fire-wall, and if your http requests have to go through a proxy, use the configure option: --with-http-proxy.
    (3) Bugs in the installation software. The compilation/installation of the individual pieces of software are generally robust. The procedure which install the totality of the package may suffer from portability bugs (it improves from version to version). To help us for support, send us an accurate description of your environment, a copy of the command you issued and of the messages returned by the computer.
    (4) Installation and operation problems with the postgres server. It exists many different possible configuration, depending if you are providing a database yet installed on your machine, using a remote access, using a yet installed Postgres package... all combinations may not have been tested!
    If the machine is stopped, the database server will be shut down and may not restart by itself. It may restart at the first request, but you may have to type: cd $PL_ROOT; make hlstart
    (5) The configuration script checks if the C++ compiler works. But either that the compiler is broken or the test incorrect, it happens that the configuration fails at that point. The c++ compiler is not required for installing Pleinpot, and you may try to solve the problem by setting: setenv CXX none and redo unix/configure>
    (6) When installing on Tru64 (OSF 5) with the native C compiler the Postgres installation may fail (bug in Postgres). The turnabout consists in defining: setenv CFLAGS "-ieee -O4 -Olimit 2000", redoing unix/configure and make. (this error should be corrected in the most recent versions of Postgres).
    (7) By default most Linux distributions do not install some header files which are needed for compiling, like for example Xlib.h. These header files are included in packages having "devel" in their name, or something similar, they can certainly be installed at any time after Linux is installed (but before configuring Pleinpot). On Mac OS X (10.3), install the developer packages Developer.mpkg and X11SDK.pkg
    (8) If after having installed Pleinpot you wish to update your operating systems or some general libraries, like libc it is wise to first save the data from the database (if you have other data than HyperLeda) and stop the database server. After upgrade you may not be able to restart the database server; in that case you will have to remove the directory hldata and reinstall the database (make install-leda and eventually reinstall your own tables). A more radical solution is (i) stop the database server as make hlstop, (ii) remove the $PL_ROOT directory and (iii) reinstall from scratch.
    (9) On Solaris, it may be necessary to define setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:/home/pleinpot/lib/postgresql (before starting the database server).
  2. If you want to use Pleinpot interactively, it is convenient to put some definitions in the .cshrc file
    . Add for example the following: 
    #-----------------------
# definitions for PLEINPOT:
#-----------------------
setenv PL_ROOT /pleinpot/
alias plstart "echo \!* >\! pls; source ${PL_ROOT}unix/plstart; /bin/rm pls"
    Or in a sh (file .bashrc) 
    #-----------------------
# definitions for PLEINPOT:
#-----------------------
export PL_ROOT=/pleinpot/
alias plstart=". ${PL_ROOT}unix/plstart.sh"
    Most important is the definition of PL_ROOT the path to the root of Pleinpot.

  3. The make command echoes each (or almost) action on the screen. It is intentional for debugging purposes, but it makes the screen hard to understand. Do not confuse echoes of commands, as '|| echo "Error failed ..."', with actual error messages. Anyway if you have doubts, redirect the output to a file and send it to me.
  4. At the installation, most compilers issue "warnings", in principle they are not important, but ...
    Identified false alerts are:
  5. There exists 3 levels of distribution of Pleinpot. The stable release is the one installed by default. You may desire to install either the Beta (possible portability problems) or Alpha (development, nightly build) most recent release. To do it, after you made the first step of the installation (unix/configure) you can type (before make install-whole): make themirror_beta; unix/configure [with your options] and then proceed to make install-whole (eventually say mirror_alpha, and/or install-pleinpot, install-whole)
    To update an alpha or beta release, always do first make themirror_alpha (or beta).

  6. The maintenance of a mirror is based on comparison of dates. Hence it is of a prime importance that the clock of the host is correctly set. Except in particular case, an error of a few minutes or even of a couple of hours has no consequence. But if the clock is in advance by about one day, the (daily) updating may not work.

  7. In order to use another Fortran compiler than the one corresponding to the command g77 or f77, type before executing configure: setenv F77 /path_to_my_preferred_f77
    The Pleinpot configure tries first to use g77 if it is present. Be aware that if you link your Pleinpot applications with other libraries (eg. Numerical Recipes) that were compiled with another Fortran compiler, you may have problems (eg. unresolved modules in principle included in the compiler's internal library).

Updating

The mirror should be updated regularly (I recommend each night). This is best done using a crontab procedure.
Note that the crontab must be owned by the user who installed the database. Running it as root will not work (root cannot start the database server if not owner of this one) and is not a good strategy in regards of security.
The crontab procedures executed on the mirrors of the home-institutions of the members of the Hypercat consortium are available in the sources of the package. See: ${PL_ROOT}unix/cron-example.

For example, the following script may be used for daily update of a particular mirror: 

#!/bin/csh
#--- cron-HOST: this routine is executed on "HOST" every morning at 3, 
#            it updates HyperLeda on the WEB
#------------------------------ PhP 23/01/98 ----r. 09/12/99 -----------------
setenv HOST nimbus
setenv PL_ROOT /data/
cd $PL_ROOT
(date ;\
 make themirror reconfigure && make install-whole ||\
 echo "Pleinpot installation on ${HOST} failed" ;\
) >&! ${PL_ROOT}public_html/cgi-bin/install/mirror.log
# ------------------- END ---------------------------------------------------
The "official" Hypercat mirrors are automatically checked from Lyon once per week in order to detect anomalies in their functioning or in the connection. Send a mail to "register" your mirror as an "official" one.

Be careful that crontab generally do not start with the same environment variables as a sh session. In particular, the environment variables PATH and LD_LIBRARY_PATH may not be set, and this may prevent some libraries or programs (eg. compilers) to be found...


HyperLeda Questions: leda@univ-lyon1.fr