Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Experts

TL;DR

This paper introduces MoSE, a mixture of stochastic experts model that captures aleatoric uncertainty in segmentation tasks by modeling multiple modes, leading to well-calibrated uncertainty estimates for ambiguous images.

Contribution

The paper proposes a novel mixture of stochastic experts model with a Wasserstein-like loss for calibrated uncertainty estimation in segmentation, addressing multi-modality in ambiguous images.

Findings

Achieves state-of-the-art performance on LIDC-IDRI and Cityscapes datasets.

Effectively models multiple segmentation modes for ambiguous inputs.

Provides well-calibrated uncertainty estimates.

Abstract

Equipping predicted segmentation with calibrated uncertainty is essential for safety-critical applications. In this work, we focus on capturing the data-inherent uncertainty (aka aleatoric uncertainty) in segmentation, typically when ambiguities exist in input images. Due to the high-dimensional output space and potential multiple modes in segmenting ambiguous images, it remains challenging to predict well-calibrated uncertainty for segmentation. To tackle this problem, we propose a novel mixture of stochastic experts (MoSE) model, where each expert network estimates a distinct mode of the aleatoric uncertainty and a gating network predicts the probabilities of an input image being segmented in those modes. This yields an efficient two-level uncertainty representation. To learn the model, we develop a Wasserstein-like loss that directly minimizes the distribution distance between the MoSE and ground truth annotations. The loss can easily integrate traditional segmentation quality measures and be efficiently optimized via constraint relaxation. We validate our method on the LIDC-IDRI dataset and a modified multimodal Cityscapes dataset. Results demonstrate that our method achieves the state-of-the-art or competitive performance on all metrics.