Leveraging neural network interatomic potentials for a foundation model of chemistry

TL;DR

This paper introduces HackNIP, a hybrid approach that uses pretrained neural network interatomic potentials to generate features for shallow models, improving property predictions in materials science.

Contribution

The study presents a novel two-stage pipeline that leverages pretrained NIPs for enhanced structure-to-property predictions, outperforming end-to-end deep neural networks in certain scenarios.

Findings

HackNIP improves data efficiency over direct fine-tuning.

Embedding depth influences the quality of features for predictions.

Hybrid approach outperforms deep models on diverse datasets.

Abstract

Large-scale foundation models, including neural network interatomic potentials (NIPs) in computational materials science, have demonstrated significant potential. However, despite their success in accelerating atomistic simulations, NIPs face challenges in directly predicting electronic properties and often require coupling to higher-scale models or extensive simulations for macroscopic properties. Machine learning (ML) offers alternatives for structure-to-property mapping but faces trade-offs: feature-based methods often lack generalizability, while deep neural networks require significant data and computational power. To address these trade-offs, we introduce HackNIP, a two-stage pipeline that leverages pretrained NIPs. This method first extracts fixed-length feature vectors (embeddings) from NIP foundation models and then uses these embeddings to train shallow ML models for downstream structure-to-property predictions. This study investigates whether such a hybridization approach, by ``hacking" the NIP, can outperform end-to-end deep neural networks, determines the dataset size at which this transfer learning approach surpasses direct fine-tuning of the NIP, and identifies which NIP embedding depths yield the most informative features. HackNIP is benchmarked on Matbench, evaluated for data efficiency, and tested on diverse tasks including \textit{ab initio}, experimental, and molecular properties. We also analyze how embedding depth impacts performance. This work demonstrates a hybridization strategy to overcome ML trade-offs in materials science, aiming to democratize high-performance predictive modeling.