A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentation

TL;DR

This review synthesizes foundational concepts and challenges in Bayesian uncertainty quantification for deep probabilistic image segmentation, aiming to improve model reliability and interpretability in real-world applications.

Contribution

It establishes a standardized framework for uncertainty modeling in segmentation, analyzes key tasks, and discusses challenges and future directions to advance the field.

Findings

Standardized theory, notation, and terminology for uncertainty in segmentation.

Identification of key challenges like uncertainty types and benchmark lack.

Guidelines for method selection, evaluation, and reproducibility.

Abstract

Advances in architectural design, data availability, and compute have driven remarkable progress in semantic segmentation. Yet, these models often rely on relaxed Bayesian assumptions, omitting critical uncertainty information needed for robust decision-making. Despite growing interest in probabilistic segmentation to address point-estimate limitations, the research landscape remains fragmented. In response, this review synthesizes foundational concepts in uncertainty modeling, analyzing how feature- and parameter-distribution modeling impact four key segmentation tasks: Observer Variability, Active Learning, Model Introspection, and Model Generalization. Our work establishes a common framework by standardizing theory, notation, and terminology, thereby bridging the gap between method developers, task specialists, and applied researchers. We then discuss critical challenges, including the nuanced distinction between uncertainty types, strong assumptions in spatial aggregation, the lack of standardized benchmarks, and pitfalls in current quantification methods. We identify promising avenues for future research, such as uncertainty-aware active learning, data-driven benchmarks, transformer-based models, and novel techniques to move from simple segmentation problems to uncertainty in holistic scene understanding. Based on our analysis, we offer practical guidelines for researchers on method selection, evaluation, reproducibility, and meaningful uncertainty estimation. Ultimately, our goal is to facilitate the development of more reliable, efficient, and interpretable segmentation models that can be confidently deployed in real-world applications.