A case study in adaptable and reusable infrastructure at the Keck Observatory Archive: VO interfaces, moving targets, and more

TL;DR

This paper details the development of adaptable, standards-compliant VO services at the Keck Observatory Archive, including spatial and temporal indexing for fast data discovery, enhancing data accessibility for diverse scientific needs.

Contribution

It introduces a flexible, open-source infrastructure with 3D spatial-temporal indexing and a configurable interface, improving data discovery and reusability at the Keck Observatory Archive.

Findings

Spatial indexing speeds up image discovery by over 100 times.

The system supports fast temporal and spatial searches for Solar System objects.

The infrastructure is adaptable for various instruments and scientific communities.

Abstract

This paper describes how the Keck Observatory Archive (KOA) is extending open source software components to develop new services. In August 2015, KOA deployed a program interface to discover public data from all instruments equipped with an imaging mode. The interface complies with version 2 of the Simple Imaging Access Protocol (SIAP), under development by the International Virtual Observatory Alliance (IVOA), which defines a standard mechanism for discovering images through spatial queries. The heart of the KOA service is an R-tree-based, database-indexing mechanism prototyped by the Virtual Astronomical Observatory (VAO) and further developed by the Montage Image Mosaic project, designed to provide fast access to large imaging data sets as a first step in creating wide-area image mosaics. The KOA service uses the results of the spatial R-tree search to create an SQLite data database for further relational filtering. The service uses a JSON configuration file to describe the association between instrument parameters and the service query parameters, and to make it applicable beyond the Keck instruments. The R-tree program was itself extended to support temporal (in addition to spatial) indexing, in response to requests from the planetary science community for a search engine to discover observations of Solar System objects. With this 3D-indexing scheme, the service performs very fast time and spatial matches between the target ephemerides, obtained from the JPL SPICE service. Our experiments indicate these matches can be more than 100 times faster than when separating temporal and spatial searches. Images of the tracks of the moving targets, overlaid with the image footprints, are computed with a new command-line visualization tool, mViewer, released with the Montage distribution. The service is currently in test and will be released in Fall 2016.