Altar: Structuring Sharable Experimental Data from Early Exploration to Publication

TL;DR

Altar is a flexible, lightweight framework that structures and manages experimental data throughout research, enhancing reproducibility and facilitating FAIR data sharing without rigid models.

Contribution

It introduces a domain-agnostic, adaptable system for organizing experimental data from early exploration to publication, integrating with existing workflows.

Findings

Supports efficient data and metadata management during active research

Enables traceability and reproducibility of experimental data

Facilitates FAIR-aligned data sharing at publication time

Abstract

Managing the data and metadata during the active development phase of an experimental project presents a significant challenge, particularly in collaborative research. This phase is frequently overlooked in Data Management Plans included in project proposals, despite its important role in ensuring reproducibility and preventing the need for retroactive reconstruction at the time of publication. Here we present Altar, a lightweight, domain-agnostic framework for structuring experimental data from the onset of a project without imposing rigid data models. Altar is built around the Sacred experiment-tracking model and captures experimental (meta)data and structures them. Parameters, metadata, curves and small files are stored in a flexible NoSQL database, while large raw data are maintained in dedicated storage and linked through unique identifiers, ensuring efficiency and traceability. This integration is composable with exiting workflows, allowing integration with minimial disruption of work habits. We document different pathways to use Altar based on users skillset (PhD students, Post-docs, Principal Investigators, Laboratory administrators, System administrators). While getting started with Altar does not require a specialized infrastructure, the framework can be easily deployed on a server and made publicly accessible when scaling up or preparing data for publication. By addressing the dynamic phase of research, Altar provides a practical bridge between exploratory experimentation and FAIR-aligned data sharing.