Benchmarking Scalable Epistemic Uncertainty Quantification in Organ Segmentation

TL;DR

This paper benchmarks various epistemic uncertainty quantification methods in organ segmentation, assessing their accuracy, calibration, and scalability to guide future development of reliable medical imaging models.

Contribution

It provides a comprehensive comparison of uncertainty estimation techniques in organ segmentation, highlighting their strengths, weaknesses, and suitability for clinical applications.

Findings

Certain methods outperform others in calibration accuracy.

Scalability varies significantly across techniques.

Some methods excel at out-of-distribution detection.

Abstract

Deep learning based methods for automatic organ segmentation have shown promise in aiding diagnosis and treatment planning. However, quantifying and understanding the uncertainty associated with model predictions is crucial in critical clinical applications. While many techniques have been proposed for epistemic or model-based uncertainty estimation, it is unclear which method is preferred in the medical image analysis setting. This paper presents a comprehensive benchmarking study that evaluates epistemic uncertainty quantification methods in organ segmentation in terms of accuracy, uncertainty calibration, and scalability. We provide a comprehensive discussion of the strengths, weaknesses, and out-of-distribution detection capabilities of each method as well as recommendations for future improvements. These findings contribute to the development of reliable and robust models that yield accurate segmentations while effectively quantifying epistemic uncertainty.