The Catalogs and Surveys Branch (CASB) of the Space Telescope Science Institute (STScI), in collaboration with a number of international astronomical institutions, is continuing with the development and creation of an archive of digitized images, and an associated catalog of stars and galaxies which cover the entire sky. These data are being made available to the astronomical community to support telescope operations and research projects.

Keywords: Astronomical Catalogs, Image Archives, Object Oriented Databases.

An important part of observational astronomy has historically been the creation of catalogs to support the operation of telescopes and their observing programs. An astronomical catalog typically contains parameters that characterize celestial objects such as position, brightness, and type of object. Modern astronomical telescopes are becoming increasingly complex and expensive to construct and operate, which demands that we optimize observing efficiency in order to maximize the scientific return of the investment. The original need for more complete all-sky catalogs was highlighted by the pointing requirements of the Hubble Space Telescope (HST). This required the construction of a catalog of 15 million stars, about ten times larger than previously existing catalogs, down to a brightness level of 15^th magnitude or 10000x fainter than stars visible to the naked eye.

The basic technique involved was to obtain photographic plates covering the entire sky, digitize these, and use image-processing techniques to identify and measure all of the stars down to the desired brightness limit. The detected objects were then stored in a custom coded database, since the relational databases of the era were unsuited to the HST-specific access requirements. This database required 1GB of storage and was at the edge of technological capability at the time catalog construction begun in 1984. A description of this catalog, GSC-I, may be found in a set of three papers (Lasker et al. 1990, Russell et al. 1990, Jenkner et al. 1990). In addition to the HST, this catalog was published and quickly became established in the operations of almost every observatory and astronomical satellite.

The original goals were merely to create a catalog to point the Hubble Space Telescope (HST). However, it quickly became clear that having access to digital images would greatly benefit not only HST operations but also the entire astronomical community. With this in mind, it was decided to develop an image archive, which would facilitate user access to these data. These images have now been distributed to the community and placed on-line at a number of institutions around the world. The availability of these images has revolutionized observational astronomical research and telescope operations.

Today, astronomers require even larger catalogs to operate ground-based telescopes and space observatories that are either under construction or being planned. As the next generation of large-aperture, new-technology telescopes becomes available, there are increasing demands for catalogs containing fainter objects to support remote or queue-scheduling capabilities. In addition, many of these telescopes have active optics, and efficient operations require convenient access to many stars within a small field of view near the target for tip/tilt corrections and for dynamic maintenance of collimation. The precision requirements of these new telescopes means that in addition to many more objects, the positions, brightness, colors and motions must also be accurately determined. Similarly, the availability of images taken at different wavelength is of immense value for astronomical research purposes.

Our overall goal is to produce digitized images of the entire sky in different optical wavelengths along with an accurate catalog of celestial objects. These data will then be made available to international astronomical observatories for telescope operations and to the astronomical community for research purposes. This will be done both by media distribution and web access to on-line archives and databases.

The key to this entire project is the availability of photographic plates that cover the entire sky. The Palomar Observatory Oschin Schmidt in California and the UK Schmidt Telescope Unit of the Anglo-Australian Observatory in Australia have provided astronomers with such a service for many years. These telescopes take photographs using special emulsions placed on 355x355 cm, 1mm thick glass. Both institutions have been performing systematic surveys of the northern and southern hemispheres respectively. A full description of the survey material available to us is listed in Table 1. These plates each cover 6.5x6.5 degrees of the sky (Figure 1) typically with 1-hour photographic exposure times. A survey of the entire sky requires approximately 1800 plates in each wavelength and 5-10 years to complete due to weather, plate quality, etc.

Two multi-channel laser-scanning microdensitometers known as the GAMMA machines (Guide Star Automated Measuring Machines) were built at STScI on Perkin-Elmer PDS substrates. The first set of plates digitized for the original GSC were scanned with 25-micron sampling (14000x14000 pixels, 1.7 arcsec size) producing a 400MB image. This sampling was selected for speed in order to meet the scheduled launch date of HST. The second epoch surveys digitized since then are scanned with a 15-micron sampling, (23040x23040 pixels, 1 arcsec size) producing a 1.1Gbyte digital image (see Figure 1). We eventually plan to replace the 25-micron scans with rescans at 15-microns. Current operations support approximately 6 scans per day.

(A) (B)

(C) (D)

At the beginning of this project in 1984, the digitized images were saved on 9-track tapes and placed in a vault. Once the decision to provide access to the data was made, these data were copied to LMSI Write-Once-Read-Many (WORM) optical media and placed in a user-area where any section of an image could be retrieved simply by placing the appropriate platter in a reader. When 8mm tapes were introduced, these were used to replace the 9-track tapes and all scans were written to 2 separate 8mm tapes, one of which is sent off-site for additional backup safety.

Once we began scanning with smaller pixel sizes, the LMSI platters (1^st generation WORM) were too small to hold an entire image and we migrated to 2^nd generation SONY WORM devices. We have since been forced to migrate away from all of these WORM media because of the increasingly difficult maintenance issues. Although the optical media are of archival quality with 30-100 year lifetimes, it is impossible to keep obsolete hardware systems running. We are currently using Rewritable Magneto-Optical media for our image archive and expect to migrate to DVD over the next 2-3 years. The archive will eventually grow to about 7000 plates that are approximately 7 Terabytes of image data.

In order to reduce the data volume to a more manageable volume, a data compression algorithm based upon an H-transform was developed (White and Postman 1992). This is a lossy technique, however it adaptively changes the scale upon which the data is smoothed such that structure on all scales is preserved. A critical examination of the compression levels showed that a 10x-compression ratio would only degrade the positional and photometric information by less than 1%, which is acceptable for all but the most demanding purposes. Even a 100x-compression was acceptable for merely providing sky images for casual purposes.

In practice, the typical user will not want to decompress an entire scan. For most purposes, a user will only wish to examine a section, a few hundred pixels. In order to do this efficiently, the original large-format image is divided up into smaller sub-images that are individually compressed and stored as separate files. The image access software will then only decompress the required files and reassemble the requested section of the image.

The compression of the digitized plate images at both these compression ratios is an on-going project at STScI. As mentioned above, these images are widely accepted as a critical resource to the astronomical community at large, including amateur uses and educational institutions of all levels. The demand has already led to the production of two separate publications, the Digital Sky Survey (DSS), a 101 CD volume all-sky image collection at 10x-compression suitable for observatories and professional use, and RealSky which is a 17 CD volume set for amateurs and educational institutions. The DSS data are also publicly available via an online CD jukebox accessed through a Web interface at STScI.

This second generation Guide Star Catalog project, GSC-II, depends on the successful operations of photographic plate scanning, image processing and object recognition techniques performed on the digital plate images, and the advanced astronomical calibrations of this data. Since we now need to compute colors and motions of the stars, it was necessary to obtain additional photographic images taken with different color filters and at widely spaced intervals.

The final production tasks to be performed are global in nature and therefore require fast cross-referencing between multiple plate object measurements and other external astronomical catalog object parameters. The COMPASS database is the key element here. By utilizing object-oriented technology to model the complex relationships between the various astronomical objects these tasks can be performed in a period of months rather than the previous cost of many years.

GSC-II will be the export of the optimal celestial object parameters, including positions, magnitudes, colors and proper motions, resulting from the systematic integration of these measured data. The catalog will contain objects 250 times fainter that GSC-I and is expected to contain about 15 billion measurements of 2 billion individual stars and galaxies (Lasker et al. 1995). This catalog will be used for operational support of HST, the GEMINI telescopes, the Italian GALILEO telescope, ESO's Very Large Telescope (VLT) as well as future space missions such as the Next Generation Space Telescope (NGST).

Although STScI began this project as part of the HST operations support, it has been supported by collaborations and cooperative arrangements with a number of institutions, each of which is involved in a different aspect of the DSS or GSC. These include Caltech's Palomar Observatory, Anglo-Australian Observatory, Royal Observatory Edinburgh, Osservatorio Astronomico di Torino, European Southern Observatory, European Space Agency (Science Division and the Space Telescope European Coordinating Facility), GEMINI telescope project, Canadian Astronomical Data Center, Centre de Données astronomiques de Strasbourg, and the National Astronomical Observatory of Japan.

The primary responsibility for the development and implementation of the GSC-II database lies with STScI, but there is significant development work in collaboration with the Osservatorio Astronomico di Torino, which is leading the Italian portion of this consortium. One of the early strategic decisions was to purchase commercial database software. Despite the limited resources and budget for the project, one of the lessons learned from GSC-I was that development and maintenance costs for a custom database eventually become excessive. Despite the improvement in Relational Databases over the last decade, it was still difficult to model the relationships between the data and perform fast and efficient queries to the data without generating indexes that become as large (or larger) than the underlying data. It quickly became clear that an Object-Oriented database was the only viable option. Another project, the Sloan Digital Sky Survey (SDSS), had also investigated many of the available OODB on the market and had selected Objectivity/DB as a result of performance testing using the GSC-I as a dataset. One of our goals was to promote database interoperability between the GSC-II and other astronomical archives, including the SDSS science archive (Szalay 1998). Consequently, after verifying that we could design an object-model that would satisfy our requirements, we began to collaborate with SDSS on the overall design of astronomical databases and obtained Objectivity/DB.

In the GSC-I database the sky is divided into almost 10000 regions in order to partition the data into manageable amounts. The goal was to have roughly the same number of objects in each region, and to enable rapid access to any section of the sky. An extension of this concept is to use the Hierarchical Triangulated Mesh (HTM), which is a quad tree based on a spatial subdivision of the celestial sphere into equal area spherical triangles (Figure 2). This code, implemented as a C++ class library, was developed by the SDSS (Sloan Digital Sky Survey) Science Archive. There is growing community consensus with most of the major astronomical database projects in adopting this as a standard method for partitioning the sky in future projects. This will allow a common identification scheme of celestial areas promoting efficient astronomical archive interoperability.

We have chosen to implement the HTM by partitioning the sky into 32768 spatial regions (HTM, 6^th level in the quad tree) and creating an Objectivity Database for each region (Figure 3). This level was chosen so that the maximum database size would not exceed the maximum file size allowed by the operating system. Within each region database we are creating several containers: one for each plate to store the measured and calibrated parameters for each source; a container for each astronomical catalog with reference sources; and an Index container which has derived multi-plate parameters and links references to the same source in the plate and catalog containers. There will typically be 5-8 observations of the same source measured on different plates, and each plate will be split up among 50-60 region databases. Depending on the nature of the query, we can easily retrieve sources grouped by plate or region after determining the appropriate list of region databases containing the sources. Access to the source parameters is achieved by iterating over the index in each region database, retrieving the derived data or using the references to directly access the raw data from the individual plates.

At STScI, image and GSC-I catalog retrieval are available by a Web interface (Figure 4). These data can be analyzed and visualized with the standard community tools such as IRAF. Several other institutions have provided similar access to the distributed data sets, one such example is the ESO (European Southern Observatory) SkyCat Tool (Albrecht 1997). SkyCat is a software tool that allows one to view images and at the same time query astronomical catalogs with visualized overlays on the images. A preliminary exported version of GSC-II has been delivered to ESO and ingested into a SkyCat network server which will be used to support the VLT (Very Large Telescope) and GEMINI telescope control systems. The first GSC-II catalog release will be delivered to ESO in FITS (Flexible Image Transport System) binary table format and is estimated to be 50 GB in size.

At present, the primary focus of the GSC-II user access development is to support catalog construction. The most critical task is object matching and cross-referencing of multiple measurements of the sources. The GET/PUT operations for this task are centered on access to the COMPASS database. Using Microsoft Developer Studio, a Visual C++ DLL was written as a middle layer between API and schema. This removes the client development from the database kernel and if chosen provides transaction control to be hidden from the API. Since many of our astronomical calibrations are written in FORTRAN, small transactions are best implemented at the API level. Within this environment, we have developed mixed-language applications with straightforward FORTRAN and C++ interfaces. This library is in the process of being extended for the global calibrations, which use statistical methods to remove systematic variations in the positions and brightness of sources.

Much of the astronomical community interest is centered on the concept of data mining. Although our current resources do not support a concentrated effort in this area, we have implicit in our design the structure to perform fast and efficient access to the data and support future development in this area. One common design pattern, "clustering for use", is key to the data mining. Cross archive access can utilize this pattern via the HTM and we continue to encourage other astronomical projects to consider this partitioning scheme in the design of access methods.

In preparation for more general user access, we have developed a 3-D JAVA visualization package, HTMBrowser, which serves as a simple query engine to the HTM C++ library with a JNI wrapper (Figure 5). The underlying graphics uses VTK (Visualization Toolkit). This package supports single coordinate queries returning the leaf-node name at the selected HTM level, corresponding to a COMPASS database, as well as area/convex intersections with a returned list. We are in the process of interfacing this to COMPASS using C++ DLL’s thus providing direct access to plate source parameters. Depending on the size of the query, the output will be a selectable screen buffered dataset and/or file. The package has been developed using platform-independent code based on JDK 1.1 or higher and ANSI standard C++. Once it has been more fully tested, the source code will be made available to other astronomical archives for research and development.

The system is split into three major parts. The plate digitization and image processing are running on Digital Alpha systems running OpenVMS. This part of our pipeline contains a great deal of legacy FORTRAN and C code that is specifically designed to run on VMS systems due to their robustness, reliability and real-time features. It is the most intensive operational procedure of the project because of the heavy costs in hardware and time to complete each task. When we started implementing the database, it also was to run under VMS and tightly integrated to the image processing. We soon faced the reality that long term VMS support was not assured, and at the same time Objectivity discontinued the VMS product release. The most feasible and cost effective solution was to move the database server to the NT operating system. As a result, there is a data transfer step between the two operating systems and the need to port much of the calibration code to NT. The scanned plate images stored as flat binary files are fed through the VMS software production pipeline. A Perl daemon then transfers these files to the NT server where the source parameters are extracted and loaded into the database as a number of plate containers within each of the region databases. The COMPASS database now contains the GSC-I plate measurements and approximately 800 2^nd-generation processed plates. We have completed development of the object-matching application and are in the process of integrating this task into the database production pipeline. The production capability for performing astronomical calibrations in the COMPASS environment is in development.

A third off-line, but equally critical, activity is the generation of a photometric catalog to support photometric calibration of the plate data. This is done by collecting CCD observations, which cover a section of each plate, and reducing this data using standard astronomical reductions tools (IRAF) on Unix platforms.

We are currently operating in three sites, STScI, USA (development, plate processing, photometric reductions, production database), Torino, Italy (development, photometric reductions, test database) and Garching, Germany (plate processing). The primary development and coordination of the operations is managed at STScI. (Figure 7)

The GSC-II project is one of the first large-scale astronomical archives in production. With the advancement in computer hardware and technology to support not only archive development, but also the potential to network these archives, large scale astronomical research that was never before possible can be performed within a time frame of months. For more information on the project, several Web pages can be viewed at http://www-gsss.stsci.edu/casbhome.html.

Albrecht, M. A., Brighton, A., Herlin, T., Biereichel, P., 1997,

"Astronomical Data Analysis Software and Systems VI",

Astronomical Society of the Pacific Conference Series, Vol.125

Barrett, P., 1995,

"Astronomical Data Analysis Software and Systems IV",

Astronomical Society of the Pacific Conference Series, Vol.77

Jenkner, H., Lasker, B.M., Sturch, C.R., McLean, B.J., Shara, M.M. and Russell, J.L., 1990 Astronomical Journal 99, 2081

Lasker, B.M., Sturch, C.R., McLean, B.J., Russell, J.L., Jenkner, H. and Shara, M.M.,

1990 Astronomical Journal 99, 2019

Lasker, B.M., McLean, B.J., Jenkner, H., Lattanzi, M.G. and Spagna, A., 1995,

"Future Possibilities for Astrometry in Space" ESA SP-379 pg.137

Russell, J.L., Lasker, B.M., McLean, B.J., Sturch, C.R. and Jenkner, H.,

1990 Astronomical Journal 99, 2059

Szalay, A. 1998

Bulletin of the American Astronomical Society, 192, 64.05

White, R.L. and Postman, M. 1992

"Digitised Optical Sky Surveys" pg.167