The long road to calibrated prediction uncertainty in computational chemistry

TL;DR

This paper reviews the current state of uncertainty quantification in computational chemistry, highlighting the lack of standard validation methods and proposing adapted tools from meteorology and machine learning for better calibration and assessment.

Contribution

It introduces adapted analysis tools for calibration and sharpness to evaluate prediction uncertainties in computational chemistry, addressing the lack of standard validation procedures.

Findings

Existing CC-UQ methods lack standardized validation.

Calibration tools from meteorology and ML are adapted for CC-UQ.

Application of these tools reveals variability in uncertainty estimates.

Abstract

Uncertainty quantification (UQ) in computational chemistry (CC) is still in its infancy. Very few CC methods are designed to provide a confidence level on their predictions, and most users still rely improperly on the mean absolute error as an accuracy metric. The development of reliable uncertainty quantification methods is essential, notably for computational chemistry to be used confidently in industrial processes. A review of the CC-UQ literature shows that there is no common standard procedure to report nor validate prediction uncertainty. I consider here analysis tools using concepts (calibration and sharpness) developed in meteorology and machine learning for the validation of probabilistic forecasters. These tools are adapted to CC-UQ and applied to datasets of prediction uncertainties provided by composite methods, Bayesian Ensembles methods, machine learning and a posteriori statistical methods.