UMA: A Family of Universal Models for Atoms

TL;DR

UMA models are a family of universal, scalable, and accurate atomic property predictors trained on half a billion structures, capable of outperforming specialized models across chemistry and materials science applications.

Contribution

The paper introduces UMA, a new family of models with a novel mixture of linear experts architecture trained on the largest atomic dataset to date, enabling high speed and accuracy.

Findings

UMA models perform on par or better than specialized models without fine-tuning.

The models demonstrate strong generalization across multiple chemical domains.

Empirical scaling laws guide model capacity and dataset size optimization.

Abstract

The ability to quickly and accurately compute properties from atomic simulations is critical for advancing a large number of applications in chemistry and materials science including drug discovery, energy storage, and semiconductor manufacturing. To address this need, Meta FAIR presents a family of Universal Models for Atoms (UMA), designed to push the frontier of speed, accuracy, and generalization. UMA models are trained on half a billion unique 3D atomic structures (the largest training runs to date) by compiling data across multiple chemical domains, e.g. molecules, materials, and catalysts. We develop empirical scaling laws to help understand how to increase model capacity alongside dataset size to achieve the best accuracy. The UMA small and medium models utilize a novel architectural design we refer to as mixture of linear experts that enables increasing model capacity without sacrificing speed. For example, UMA-medium has 1.4B parameters but only ~50M active parameters per atomic structure. We evaluate UMA models on a diverse set of applications across multiple domains and find that, remarkably, a single model without any fine-tuning can perform similarly or better than specialized models. We are releasing the UMA code, weights, and associated data to accelerate computational workflows and enable the community to continue to build increasingly capable AI models.