Position: Epistemic uncertainty estimation methods are fundamentally incomplete

TL;DR

This paper argues that current methods for estimating epistemic uncertainty in supervised learning are fundamentally incomplete, as they are contaminated by bias and only capture partial variance sources, limiting their reliability in safety-critical applications.

Contribution

The paper demonstrates the fundamental limitations of existing second-order methods in disentangling and accurately estimating epistemic uncertainty, highlighting the need for more comprehensive approaches.

Findings

Bias contaminates uncertainty estimates, overestimating aleatoric and underestimating epistemic uncertainty.

Existing methods only partially capture variance-driven epistemic uncertainty.

Current estimates require careful interpretation and acknowledgment of limitations in safety-critical contexts.

Abstract

Identifying and disentangling sources of predictive uncertainty is essential for trustworthy supervised learning. We argue that widely used second-order methods that disentangle aleatoric and epistemic uncertainty are fundamentally incomplete. First, we show that unaccounted bias contaminates uncertainty estimates by overestimating aleatoric (data-related) uncertainty and underestimating the epistemic (model-related) counterpart, leading to incorrect uncertainty quantification. Second, we demonstrate that existing methods capture only partial contributions to the variance-driven part of epistemic uncertainty; different approaches account for different variance sources, yielding estimates that are incomplete and difficult to interpret. Together, these results highlight that current epistemic uncertainty estimates can only be used in safety-critical and high-stakes decision-making when limitations are fully understood by end users and acknowledged by AI developers.