Uncertainty in Physics and AI: Taxonomy, Quantification, and Validation

TL;DR

This paper offers a comprehensive overview of uncertainty quantification in physics-related machine learning, introducing a taxonomy, interpretation guidelines, and validation tools to improve scientific reliability.

Contribution

It presents a unified taxonomy of uncertainty, clarifies interpretative frameworks, and discusses validation methods for probabilistic statements in physics ML.

Findings

Introduces a structured taxonomy of uncertainty types.

Clarifies interpretation of predictive and inference uncertainties.

Discusses validation tools like coverage, calibration, and scoring rules.

Abstract

Reliable uncertainty quantification is essential for the use of machine learning in physics, where scientific discoveries depend on validated probabilistic statements. We provide a structured overview of uncertainty quantification in ML for physics, introducing a unified taxonomy of uncertainty and clarifying the interpretation of predictive and inference uncertainties across frequentist and Bayesian frameworks. We discuss principled validation tools, including coverage, calibration, bias tests, and proper scoring rules, and illustrate them with simple regression and classification examples.