Enhancing Machine Learning Potentials through Transfer Learning across Chemical Elements

TL;DR

This paper introduces transfer learning for Machine Learning Potentials across similar chemical elements, significantly reducing data requirements and improving stability and transferability in simulations.

Contribution

The study demonstrates that transfer learning between chemically similar elements enhances MLP training efficiency and stability, especially with limited data, advancing the development of accurate potentials.

Findings

Transfer learning outperforms training from scratch in force prediction.

Transfer learning improves stability and temperature transferability.

Benefits are more pronounced with smaller datasets.

Abstract

Machine Learning Potentials (MLPs) can enable simulations of ab initio accuracy at orders of magnitude lower computational cost. However, their effectiveness hinges on the availability of considerable datasets to ensure robust generalization across chemical space and thermodynamic conditions. The generation of such datasets can be labor-intensive, highlighting the need for innovative methods to train MLPs in data-scarce scenarios. Here, we introduce transfer learning of potential energy surfaces between chemically similar elements. Specifically, we leverage the trained MLP for silicon to initialize and expedite the training of an MLP for germanium. Utilizing classical force field and ab initio datasets, we demonstrate that transfer learning surpasses traditional training from scratch in force prediction, leading to more stable simulations and improved temperature transferability. These advantages become even more pronounced as the training dataset size decreases. The out-of-target property analysis shows that transfer learning leads to beneficial but sometimes adversarial effects. Our findings demonstrate that transfer learning across chemical elements is a promising technique for developing accurate and numerically stable MLPs, particularly in a data-scarce regime.