Pre-training, fine-tuning, and distillation (PFD): Automatically generating machine learning force fields from universal models

TL;DR

This paper introduces the PFD workflow that efficiently generates material-specific machine learning force fields from universal models, achieving first-principles accuracy and fast inference suitable for large-scale simulations.

Contribution

It presents a novel PFD workflow combining pre-training, fine-tuning, and distillation to produce accurate, fast force fields from universal models with less training data.

Findings

Force fields with first-principles accuracy generated from universal models.

Reduced training data requirements by one to two orders of magnitude.

Enhanced inference speed suitable for large-scale molecular simulations.

Abstract

Universal force fields generalizable across the periodic table represent a new trend in computational materials science. However, the applications of universal force fields in material simulations are limited by their slow inference speed and the lack of first-principles accuracy. Instead of building a single model simultaneously satisfying these characteristics, a strategy that quickly generates material-specific models from the universal model may be more feasible. Here, we propose a new workflow pattern, PFD (Pre-training, Fine-tuning, and Distillation), which automatically generates machine-learning force fields for specific materials from a pre-trained universal model through fine-tuning and distillation. By fine-tuning the pre-trained model, our PFD workflow generates force fields with first-principles accuracy while requiring one to two orders of magnitude less training data compared to traditional methods. The inference speed of the generated force field is further improved through distillation, meeting the requirements of large-scale molecular simulations. Comprehensive testing across diverse materials including complex systems, such as amorphous carbon, interface, etc., reveals marked enhancements in training efficiency, which suggests the PFD workflow a practical and reliable approach for force field generation in computational material sciences.