Provenance tracking in the LHCb software

TL;DR

This paper introduces an integrated provenance tracking service within LHCb's scientific software, capturing metadata to enhance reproducibility of datasets without requiring additional tools or training.

Contribution

It presents a novel, integrated provenance tracking solution embedded in existing software, enabling dataset reproducibility even if original code is changed or unavailable.

Findings

Provenance data is automatically stored within output files.

The system allows reproducing datasets despite code alterations.

Implementation details and practical examples are provided.

Abstract

Even though computational reproducibility is widely accepted as necessary for research validation and reuse, it is often not considered during the research process. This is because reproducibility tools are typically stand-alone and require additional training to be employed. In this article, we present a solution to foster reproducibility, which is integrated within existing scientific software that is actively used in the LHCb collaboration. Our provenance tracking service captures metadata of a dataset, which is then saved inside the output data file on the disk. The captured information allows a complete understanding of how the file was produced and enables a user to reproduce the dataset, even when the original input code (that was used to initially produce the dataset) is altered or lost. This article describes the implementation of the service and gives examples of its application.