pynxtools: A Python framework for generating and validating NeXus files in experimental data workflows

TL;DR

pynxtools is a Python framework that standardizes and validates scientific data conversion to the NeXus format, promoting FAIR principles and interoperability across diverse experimental techniques.

Contribution

It introduces a modular, plugin-based Python framework with a CLI for converting and validating experimental data into standardized NeXus files, ensuring data consistency and compliance.

Findings

Supports multiple scientific domains like microscopy and spectroscopy.

Enables integration with research data management systems.

Ensures data correctness and NeXus compliance.

Abstract

Scientific data across physics, materials science, and materials engineering often lacks adherence to FAIR principles (Barker et al., 2022; Jacobsen et al., 2020; M. D. Wilkinson et al., 2016; S. R. Wilkinson et al., 2025) due to incompatible instrument-specific formats and diverse standardization practices. pynxtools is a Python software development framework with a command line interface (CLI) that standardizes data conversion for scientific experiments in materials science to the NeXus format (Klosowski et al., 1997; K\"onnecke, 2006; K\"onnecke et al., 2015) across diverse scientific domains. NeXus defines data storage specifications for different experimental techniques through application definitions. pynxtools provides a fixed, versioned set of NeXus application definitions that ensures convergence and alignment in data specifications across, among others, atom probe tomography, electron microscopy, optical spectroscopy, photoemission spectroscopy, scanning probe microscopy, and X-ray diffraction. Through its modular plugin architecture pynxtools provides conversion of data and metadata from instruments and electronic lab notebooks to these unified definitions, while performing validation to ensure data correctness and NeXus compliance. pynxtools can be integrated directly into Research Data Management Systems (RDMS) to facilitate parsing and normalization. We detail one example for the RDM system NOMAD. By simplifying the adoption of NeXus, the framework enables true data interoperability and FAIR data management across multiple experimental techniques.