Black-Box Uncertainty Estimation for Deep Learning Models in Atomistic Simulations

TL;DR

This paper presents a black-box ensemble-based uncertainty quantification method for atomistic neural networks that effectively detects out-of-distribution data and correlates with model convergence, without altering the original model architecture.

Contribution

The authors introduce a black-box ensemble approach for uncertainty estimation in atomistic neural networks that does not require modifying the existing model architecture.

Findings

The method accurately detects out-of-distribution molecular data.

Scaled uncertainty signals correlate with model convergence during training.

The approach is robust across different temperature conditions for Na and Al systems.

Abstract

We analyze an ensemble-based approach for uncertainty quantification (UQ) in atomistic neural networks. This method generates an epistemic uncertainty signal without requiring changes to the underlying multi-headed regression neural network architecture, making it suitable for sealed or black-box models. We apply this method to molecular systems, specifically sodium (Na) and aluminum (Al), under various temperature conditions. By scaling the uncertainty signal, we account for heteroscedasticity in the data. We demonstrate the robustness of the scaled UQ signal for detecting out-of-distribution (OOD) behavior in several scenarios. This UQ signal also correlates with model convergence during training, providing an additional tool for optimizing the training process.