Running a distributed virtual observatory: US Virtual Astronomical Observatory operations

TL;DR

This paper discusses how the US Virtual Astronomical Observatory manages distributed data and resources, addressing challenges like data volume and decentralization to provide effective tools and services for astronomers.

Contribution

It presents strategies and implementations for operating a distributed virtual observatory, including monitoring, caching, and distributed storage, extending international VO protocols.

Findings

Successful implementation of continuous monitoring and validation

Enhanced data access through caching and distributed storage

Operational strategies for managing distributed resources

Abstract

Operation of the US Virtual Astronomical Observatory shares some issues with modern physical observatories, e.g., intimidating data volumes and rapid technological change, and must also address unique concerns like the lack of direct control of the underlying and scattered data resources, and the distributed nature of the observatory itself. In this paper we discuss how the VAO has addressed these challenges to provide the astronomical community with a coherent set of science-enabling tools and services. The distributed nature of our virtual observatory-with data and personnel spanning geographic, institutional and regime boundaries-is simultaneously a major operational headache and the primary science motivation for the VAO. Most astronomy today uses data from many resources. Facilitation of matching heterogeneous datasets is a fundamental reason for the virtual observatory. Key aspects of our approach include continuous monitoring and validation of VAO and VO services and the datasets provided by the community, monitoring of user requests to optimize access, caching for large datasets, and providing distributed storage services that allow user to collect results near large data repositories. Some elements are now fully implemented, while others are planned for subsequent years. The distributed nature of the VAO requires careful attention to what can be a straightforward operation at a conventional observatory, e.g., the organization of the web site or the collection and combined analysis of logs. Many of these strategies use and extend protocols developed by the international virtual observatory community.