Evolution of Data Formats in Very-High-Energy Gamma-ray Astronomy

TL;DR

This paper reviews the evolution of data formats in very-high-energy gamma-ray astronomy, emphasizing the shift from proprietary to open, standardized formats to facilitate data sharing, reproducibility, and integration across instruments.

Contribution

It provides a comprehensive overview of the development of standardized data formats in gamma-ray astronomy, highlighting their role in enabling open science and data interoperability.

Findings

Standardized data formats improve data integration across instruments.

Open data formats facilitate reproducible research in gamma-ray astronomy.

Prototypical formats have been successfully used in recent publications.

Abstract

Most major scientific results produced by ground-based gamma-ray telescopes in the last 30 years have been obtained by expert members of the collaborations operating these instruments. This is due to the proprietary data and software policies adopted by these collaborations. However, the advent of the next generation of telescopes and their operation as observatories open to the astronomical community, along with a generally increasing demand for open science, confront gamma-ray astronomers with the challenge of sharing their data and analysis tools. As a consequence, in the last few years, the development of open-source science tools has progressed in parallel with the endeavour to define a standardised data format for astronomical gamma-ray data. The latter constitutes the main topic of this review. Common data specifications provide equally important benefits to the current and future generation of gamma-ray instruments: they allow the data from different instruments, including legacy data from decommissioned telescopes, to be easily combined and analysed within the same software framework. In addition, standardised data accessible to the public, and analysable with open-source software, grant fully-reproducible results. In this article we provide an overview of the evolution of the data format for gamma-ray astronomical data, focusing on its progression from private and diverse specifications to prototypical open and standardised ones. The latter have already been successfully employed in a number of publications paving the way to the analysis of data from the next generation of gamma-ray instruments, and to an open and reproducible way of conducting gamma-ray astronomy.