Evaluation of Provenance Serialisations for Astronomical Provenance

TL;DR

This study compares turtle and JSON provenance serialisations for astronomical data, evaluating their efficiency in storage, upload, and querying within different database systems to inform best practices for large-scale astronomical surveys.

Contribution

It provides an empirical comparison of turtle and JSON provenance serialisations using representative database systems, highlighting their relative efficiencies for storage and complex querying.

Findings

Turtle serialisation is more efficient for storage and small, simple queries.

JSON serialisation performs better for complex pattern-matching queries.

Both serialisations have similar query accuracy.

Abstract

Provenance data from astronomical pipelines are instrumental in establishing trust and reproducibility in the data processing and products. In addition, astronomers can query their provenance to answer questions routed in areas such as anomaly detection, recommendation, and prediction. The next generation of astronomical survey telescopes such as the Vera Rubin Observatory or Square Kilometre Array, are capable of producing peta to exabyte scale data, thereby amplifying the importance of even small improvements to the efficiency of provenance storage or querying. In order to determine how astronomers should store and query their provenance data, this paper reports on a comparison between the turtle and JSON provenance serialisations. The triple store Apache Jena Fuseki and the graph database system Neo4j were selected as representative database management systems (DBMS) for turtle and JSON, respectively. Simulated provenance data was uploaded to and queried over each DBMS and the metrics measured for comparison were the accuracy and timing of the queries as well as the data upload times. It was found that both serialisations are competent for this purpose, and both have similar query accuracy. The turtle provenance was found to be more efficient at storing and uploading the data. Regarding queries, for small datasets ($<$5MB) and simple information retrieval queries, the turtle serialisation was also found to be more efficient. However, queries for JSON serialised provenance were found to be more efficient for more complex queries which involved matching patterns across the DBMS, this effect scaled with the size of the queried provenance.