Comparative study of ensemble-based uncertainty quantification methods for neural network interatomic potentials

TL;DR

This study evaluates ensemble-based uncertainty quantification methods for neural network interatomic potentials, revealing limitations in their reliability for system-level predictions, especially in out-of-distribution scenarios.

Contribution

It systematically compares multiple ensemble-based methods for uncertainty quantification in neural network potentials and assesses their effectiveness in both in-distribution and out-of-distribution regimes.

Findings

Uncertainty estimates can plateau or decrease as errors increase in OOD settings.

Current methods have fundamental limitations in reliably indicating accuracy for extrapolative predictions.

Caution is advised when using predictive uncertainty as a proxy for accuracy in large-scale simulations.

Abstract

Machine learning interatomic potentials (MLIPs) enable atomistic simulations with near first-principles accuracy at substantially reduced computational cost, making them powerful tools for large-scale materials modeling. The accuracy of MLIPs is typically validated on a held-out dataset of \emph{ab initio} energies and atomic forces. However, accuracy on these small-scale properties does not guarantee reliability for emergent, system-level behavior -- precisely the regime where atomistic simulations are most needed, but for which direct validation is often computationally prohibitive. As a practical heuristic, predictive precision -- quantified as inverse uncertainty -- is commonly used as a proxy for accuracy, but its reliability remains poorly understood, particularly for system-level predictions. In this work, we systematically assess the relationship between predictive precision and accuracy in both in-distribution (ID) and out-of-distribution (OOD) regimes, focusing on ensemble-based uncertainty quantification methods for neural network potentials, including bootstrap, dropout, random initialization, and snapshot ensembles. We use held-out cross-validation for ID assessment and calculate cold curve energies and phonon dispersion relations for OOD testing. These evaluations are performed across various carbon allotropes as representative test systems. We find that uncertainty estimates can behave counterintuitively in OOD settings, often plateauing or even decreasing as predictive errors grow. These results highlight fundamental limitations of current uncertainty quantification approaches and underscore the need for caution when using predictive precision as a stand-in for accuracy in large-scale, extrapolative applications.