Rethinking Uncertainty Quantification and Entanglement in Image Segmentation

TL;DR

This paper empirically studies how different uncertainty quantification methods in image segmentation interact and entangle, proposing metrics and analyzing their performance across tasks to improve interpretability and safety.

Contribution

It provides a comprehensive empirical analysis of AU-EU model combinations, introduces a metric for entanglement, and evaluates their effectiveness in various downstream tasks.

Findings

Ensembles show lower entanglement and better out-of-distribution detection.

Softmax/SSN methods perform well in ambiguity modeling and calibration.

Probabilistic UNets are less effective in disentangling uncertainties.

Abstract

Uncertainty quantification (UQ) is crucial in safety-critical applications such as medical image segmentation. Total uncertainty is typically decomposed into data-related aleatoric uncertainty (AU) and model-related epistemic uncertainty (EU). Many methods exist for modeling AU (such as Probabilistic UNet, Diffusion) and EU (such as ensembles, MC Dropout), but it is unclear how they interact when combined. Additionally, recent work has revealed substantial entanglement between AU and EU, undermining the interpretability and practical usefulness of the decomposition. We present a comprehensive empirical study covering a broad range of AU-EU model combinations, propose a metric to quantify uncertainty entanglement, and evaluate both across downstream UQ tasks. For out-of-distribution detection, ensembles exhibit consistently lower entanglement and superior performance. For ambiguity modeling and calibration the best models are dataset-dependent, with softmax/SSN-based methods performing well and Probabilistic UNets being less entangled. A softmax ensemble fares remarkably well on all tasks. Finally, we analyze potential sources of uncertainty entanglement and outline directions for mitigating this effect.