Synthetic pre-training for neural-network interatomic potentials

TL;DR

This paper introduces a pre-training approach using synthetic atomistic data generated by existing ML potentials to improve the training efficiency and accuracy of neural-network interatomic potentials, especially for small quantum-mechanical datasets.

Contribution

The authors propose a novel synthetic data pre-training method for neural-network interatomic potentials, enhancing their accuracy and stability with limited quantum data.

Findings

Pre-training with synthetic data improves model accuracy.

Fine-tuning on small quantum datasets is more effective.

Feasibility demonstrated for carbon neural potentials.

Abstract

Machine learning (ML) based interatomic potentials have transformed the field of atomistic materials modelling. However, ML potentials depend critically on the quality and quantity of quantum-mechanical reference data with which they are trained, and therefore developing datasets and training pipelines is becoming an increasingly central challenge. Leveraging the idea of "synthetic" (artificial) data that is common in other areas of ML research, we here show that synthetic atomistic data, themselves obtained at scale with an existing ML potential, constitute a useful pre-training task for neural-network interatomic potential models. Once pre-trained with a large synthetic dataset, these models can be fine-tuned on a much smaller, quantum-mechanical one, improving numerical accuracy and stability in computational practice. We demonstrate feasibility for a series of equivariant graph-neural-network potentials for carbon, and we carry out initial experiments to test the limits of the approach.