Metatensor and metatomic: foundational libraries for interoperable atomistic machine learning

TL;DR

This paper introduces two software libraries, metatensor and metatomic, designed to enable seamless data sharing, model management, and interoperability between atomistic machine learning frameworks and traditional simulation tools, addressing key integration challenges.

Contribution

The paper presents novel libraries that facilitate interoperability and data management for atomistic ML, bridging the gap between ML frameworks and simulation software.

Findings

Metatensor enables multi-platform, multi-language array manipulation with geometric metadata.

Metatomic provides portable storage for atomistic ML models and metadata.

The libraries improve integration and data sharing between ML and atomistic simulation tools.

Abstract

Incorporation of machine learning (ML) techniques into atomic-scale modeling has proven to be an extremely effective strategy to improve the accuracy and reduce the computational cost of simulations. It also entails conceptual and practical challenges, as it involves combining very different mathematical foundations, as well as software ecosystems that are very well developed in their own right, but do not share many commonalities. To address these issues and facilitate the adoption of ML in atomistic simulations, we introduce two dedicated software libraries. The first one, metatensor, provides multi-platform and multi-language storage and manipulation of arrays with many potentially sparse indices, designed from the ground up for atomistic ML applications. By combining the actual values with metadata that describes their nature and that facilitates the handling of geometric information and gradients with respect to the atomic positions, metatensor provides a common framework to enable data sharing between ML software -- typically written in Python -- and established atomistic modeling tools -- typically written in Fortran, C or C++. The second library, metatomic, provides an interface to store an atomistic ML model and metadata about this model in a portable way, facilitating the implementation, training and distribution of models, and their use across different simulation packages. We showcase a growing ecosystem of tools, including low-level libraries, training utilities, and interfaces with existing software packages that demonstrate the effectiveness of metatensor and metatomic in bridging the gap between traditional simulation software and modern ML frameworks.