A Graph Neural Network for the Era of Large Atomistic Models

TL;DR

This paper introduces DPA3, a scalable graph neural network model designed for large atomistic models, demonstrating improved generalization and accuracy across diverse materials and molecules, aligning with the scaling law.

Contribution

The paper presents DPA3, a novel multi-layer graph neural network that scales effectively with model size and training data, achieving state-of-the-art zero-shot generalization in atomistic modeling.

Findings

DPA3's generalization error follows the scaling law.

Stacking layers enhances model scalability.

DPA3 outperforms benchmarks in diverse atomistic tasks.

Abstract

Foundation models, or large atomistic models (LAMs), aim to universally represent the ground-state potential energy surface (PES) of atomistic systems as defined by density functional theory (DFT). The scaling law is pivotal in the development of large models, suggesting that their generalizability in downstream tasks consistently improves with increased model size, expanded training datasets, and larger computational budgets. In this study, we present DPA3, a multi-layer graph neural network founded on line graph series (LiGS), designed explicitly for the era of LAMs. We demonstrate that the generalization error of the DPA3 model adheres to the scaling law. The scalability in the number of model parameters is attained by stacking additional layers within DPA3. Additionally, the model employs a dataset encoding mechanism that decouples the scaling of training data size from the model size within its multi-task training framework. When trained as problem-oriented potential energy models, the DPA3 model exhibits superior accuracy in the majority of benchmark cases, encompassing systems with diverse features, including molecules, bulk materials, surface and cluster catalysts, two-dimensional materials, and battery materials. When trained as a LAM on the OpenLAM-v1 dataset, the DPA-3.1-3M model exhibits lowest overall zero-shot generalization error across 12 downstream tasks spanning a diverse array of research domains. This performance suggests superior accuracy as an out-of-the-box potential model, requiring minimal fine-tuning data for downstream scientific applications.