A recipe for scalable attention-based MLIPs: unlocking long-range accuracy with all-to-all node attention

TL;DR

This paper introduces AllScAIP, an attention-based MLIP model that effectively captures long-range interactions in large systems, achieving state-of-the-art accuracy and enabling stable molecular dynamics simulations.

Contribution

The paper presents AllScAIP, a scalable, energy-conserving MLIP with all-to-all node attention that improves long-range interaction modeling in large systems.

Findings

AllScAIP achieves state-of-the-art accuracy on molecular energy and force predictions.

All-to-all attention is crucial for modeling long-range interactions in large systems.

The model enables stable, long-timescale molecular dynamics simulations.

Abstract

Machine-learning interatomic potentials (MLIPs) have advanced rapidly, with many top models relying on strong physics-based inductive biases. However, as models scale to larger systems like biomolecules and electrolytes, they struggle to accurately capture long-range (LR) interactions, leading current approaches to rely on explicit physics-based terms or components. In this work, we propose AllScAIP, a straightforward, attention-based, and energy-conserving MLIP model that scales to O(100 million) training samples. It addresses the long-range challenge using an all-to-all node attention component that is data-driven. Extensive ablations reveal that in low-data/small-model regimes, inductive biases improve sample efficiency. However, as data and model size scale, these benefits diminish or even reverse, while all-to-all attention remains critical for capturing LR interactions. Our model achieves state-of-the-art energy/force accuracy on molecular systems, as well as a number of physics-based evaluations (OMol25), while being competitive on materials (OMat24) and catalysts (OC20). Furthermore, it enables stable, long-timescale MD simulations that accurately recover experimental observables, including density and heat of vaporization predictions.