Extrapolating from neural network models: a cautionary tale

TL;DR

This paper introduces three methods to estimate lower bounds of error bars for neural network extrapolations, demonstrating their application to nuclear mass predictions and emphasizing the importance of uncertainty quantification in extrapolation tasks.

Contribution

It proposes three novel methods for estimating lower bounds of extrapolation errors in neural networks and benchmarks their performance in nuclear physics applications.

Findings

The methods provide conservative error estimates for neural network extrapolations.

Error bars can identify regions where neural network predictions become unreliable.

Application to nuclear mass data demonstrates practical utility of the methods.

Abstract

We present three different methods to estimate error bars on the predictions made using a neural network. All of them represent lower bounds for the extrapolation errors. For example, we did not include an analysis on robustness against small perturbations of the input data. At first, we illustrate the methods through a simple toy model, then, we apply them to some realistic cases related to nuclear masses. By using theoretical data simulated either with a liquid-drop model or a Skyrme energy density functional, we benchmark the extrapolation performance of the neural network in regions of the Segr\`e chart far away from the ones used for the training and validation. Finally, we discuss how error bars can help identifying when the extrapolation becomes too uncertain and thus unreliable