leda Examples of pipeline processing site

| HyperLeda home | Documentation | Search by name | Search near a position | Define a sample | SQL Search |

This page presents examples of "on the fly" processing of FITS astronomical data available on the WEB using the HyperLeda pipeline tools.

Processing of a long slit spectrum archived in HyperLeda

HyperLeda contains a FITS archive whith many "reference" datasets.

In many cases, raw data are archived and a consisten description of the file (FITS header) allows to process them on the fly. This allows to make customized parameter extractions.

Lets consider as an example this file: L93111HP1/00147, a spectrum of NGC7785.

This first command makes a "pre-view" display of the image.
o=fa:L93111HP1/00147&z=vg

The following command executes: The flat-field correction, a removal of skipes (cosmics), the resampling in wavelength, subtraction of the sky, display of the 2D corrected image, extraction of the central 10 arcsec and finally display of the final 1D spectrum.
o=fa:L93111HP1/00147&z=ffc|spk[2]|wrs|sks|vg|xto[a,10]|vs

The commands consists in a series of pairs parameters=values separated by the sign &. In these example the parameters are o and z respectively corresponding to (o) the address of the data injected in the pipeline and (z) the suite of commands executed on the pipeline.
The commands executed on the pipeline are separated with the character | and have in general arguments given within [ ]. These arguments can be omitted when they have a default value.
In this example the address of the data is fa:... (for FITS archive). The next example will show other possible address.
The pipeline is executed on a single machine, but a series of pipelines on differents machine can be attached together. Possibly, arguments of commands on the pipeline may also be pipelines.
The type of output is controlled by the parameter a. The next command will download the result of the pipeline into a FITS file (the most useful case).
o=fa:L93111HP1/00147&a=mime%3Aapplication/fits

One more example: Take the spectrum of a star observed with ELODIE (R=42000), show it, calibrate in physical flux, show it, broaden it to a velocity dispersion of 200 km/s and red-shift it to 5000 km/s, show it.
o=fa:LH_ELODIE/00467&z=vs|fca[1,phy]|vs|vsg[5000,200]|vs

Access to various resources: DSS, DENIS, 2MASS, HIPASS

Here are examples of commands to access data from different surveys, see serveur_pixel.html (the commands below only make a pre-views but pipelines may be applied...).

Evolutive template spectra

Remote access to data and processing is of course not restricted to observation. The next example shows how to synthetize a stellar population with PEGASE.
o=ets:T1_L1_I5,3,0.005,1,0.7&z=xtc[m,63,m,63]|wrs[1,4600,6000,2]|vs
This command generates an evolutive spectrum, extracts one age, and rebins it in the wavelength range 4600 to 600 nm with a step of 0.2 nm.

SDSS spectra

The spectra of the early release of the Sloan Digital Sky survey are available from: http://archive.stsci.edu/sdss/
For example, the next address returns the FITS spectra of a galaxy (if you click on the link your browser will probably display the binary content of the FITS file... you may download the data to a file and use a program reading FITS)
http://sdssdp7.fnal.gov/data/dp7.b/spectro/1d_10/0266/1d/spSpec-51602-0266-001.fit

It is easy to inject this file into the processing pipeline. For example the following command takes this spectra, display it as it is (ie. sampled in log(wavelength))
fG.cgi?c=i&o=http://sdssdp7.fnal.gov/data/dp7.b/spectro/1d_10/0266/1d/spSpec-51602-0266-001.fit&z=xtc[m,1,m,1]|vs
And the next command resample this spectrum in linear wavelength in the range 450 to 600 nm with a step 0f 0.1 nm, and show the result:
fG.cgi?c=i&0=http://sdssdp7.fnal.gov/data/dp7.b/spectro/1d_10/0266/1d/spSpec-51602-0266-001.fit&z=xtc[m,1,m,1]|wrs[1,4500,6000,1]|vs

How to use online pipelines?

This page has shown examples which are executed from a WEB browser. It has also illustrated how a pipeline can be edited (eg. L93111HP1/00147). But it is probably not what you need at the end...
The HTML pipeline-editor and the documentation are well suited for human reading, but in an astronomical project you will wish to execute the pipelines within your own programs. The result of a pipeline processing is a FITS file that you can read as a normal disk file. Modern FITS readers, as CFITSIO, can directly read FITS files over internet and therefore can execute the pipelines designed as cgi.
This gives an easy method to use distributed computing resources accessing remote data. This gives you also convenient solutions to distribute your data or analysis methods (a lot of code and packages are available to reduce significantly the effort of development).

Where to find FITS data I can use?

This question is called: "data-mining". Databases as HyperLeda and NED keep internal archive which provide "reference" data. For example, you can find the data stored in HyperLeda for NGC7785 as:
fG.cgi?n=11&o=NGC7785 The observatory archives give access to pointed observations, and specialized archives distribute large (space) missions and surveys. All data-center offer links to these resources, and the question of the data-mining is in the core of the Virtual Observatory and evolves rapidly.


HyperLeda Questions: leda@univ-lyon1.fr