Towards a Provenance Management System for Astronomical Observatories

TL;DR

This paper develops a provenance management system tailored for astronomical observatories, integrating use cases, data models, and tools to capture, store, and visualize provenance information for complex data pipelines.

Contribution

It introduces a comprehensive provenance management framework for astronomy, combining existing tools and models to handle diverse data provenance needs.

Findings

Prototypes successfully capture and visualize provenance data.

The system supports both existing data and new data pipelines.

A data model aligned with IVOA standards was established.

Abstract

We present here a provenance management system adapted to astronomical projects needs. We collected use cases from various astronomy projects and defined a data model in the ecosystem developed by the IVOA (International Virtual Observatory Alliance). From those use cases, we observed that some projects already have data collections generated and archived, from which the provenance has to be extracted (provenance "on top"), and some projects are building complex pipelines that automatically capture provenance information during the data processing (capture "inside"). Different tools and prototypes have been developed and tested to capture, store, access and visualize the provenance information, which participate to the shaping of a full provenance management system able to handle detailed provenance information.