Transfer learning on large datasets for the accurate prediction of material properties

TL;DR

This paper demonstrates that transfer learning with graph neural networks trained on large crystal datasets can effectively extend predictions to different density functionals, reducing data needs and improving accuracy in material property prediction.

Contribution

It introduces a transfer learning approach that leverages large datasets for different density functionals, enhancing material property predictions with fewer data and analyzing dataset size effects.

Findings

Pre-training reduces dataset size needed for accuracy.

Error scales linearly with dataset size on a log-log scale.

Additional low-cost functional calculations improve predictions.

Abstract

Graph neural networks trained on large crystal structure databases are extremely effective in replacing ab initio calculations in the discovery and characterization of materials. However, crystal structure datasets comprising millions of materials exist only for the Perdew-Burke-Ernzerhof (PBE) functional. In this work, we investigate the effectiveness of transfer learning to extend these models to other density functionals. We show that pre-training significantly reduces the size of the dataset required to achieve chemical accuracy and beyond. We also analyze in detail the relationship between the transfer-learning performance and the size of the datasets used for the initial training of the model and transfer learning. We confirm a linear dependence of the error on the size of the datasets on a log-log scale, with a similar slope for both training and the pre-training datasets. This shows that further increasing the size of the pre-training dataset, i.e. performing additional calculations with a low-cost functional, is also effective, through transfer learning, in improving machine-learning predictions with the quality of a more accurate, and possibly computationally more involved functional. Lastly, we compare the efficacy of interproperty and intraproperty transfer learning.