Fine-tuning foundation models of materials interatomic potentials with frozen transfer learning

TL;DR

This paper shows that foundation models for materials interatomic potentials can be significantly improved to achieve chemical accuracy through transfer learning with partially frozen weights, reducing data and computational requirements.

Contribution

It introduces a transfer learning approach with frozen weights to enhance foundation models' accuracy and efficiency in materials simulations.

Findings

Frozen transfer learning with 10-20% of data achieves similar accuracy to models trained from scratch.

Transfer learned potentials serve as efficient surrogates, reducing computational costs.

The workflow improves data and computational efficiency in materials simulations.

Abstract

Machine-learned interatomic potentials are revolutionising atomistic materials simulations by providing accurate and scalable predictions within the scope covered by the training data. However, generation of an accurate and robust training data set remains a challenge, often requiring thousands of first-principles calculations to achieve high accuracy. Foundation models have started to emerge with the ambition to create universally applicable potentials across a wide range of materials. While foundation models can be robust and transferable, they do not yet achieve the accuracy required to predict reaction barriers, phase transitions, and material stability. This work demonstrates that foundation model potentials can reach chemical accuracy when fine-tuned using transfer learning with partially frozen weights and biases. For two challenging datasets on reactive chemistry at surfaces and stability and elastic properties of tertiary alloys, we show that frozen transfer learning with 10-20% of the data (hundreds of datapoints) achieves similar accuracies to models trained from scratch (on thousands of datapoints). Moreover, we show that an equally accurate, but significantly more efficient surrogate model can be built using the transfer learned potential as the ground truth. In combination, we present a simulation workflow for machine learning potentials that improves data efficiency and computational efficiency.