Deterministic Mode Proposals: An Efficient Alternative to Generative Sampling for Ambiguous Segmentation

TL;DR

This paper proposes a deterministic mode proposal framework for ambiguous segmentation tasks, offering a faster and more effective alternative to generative sampling by directly generating multiple likely outcomes in a single pass.

Contribution

It introduces a novel deterministic approach for mode proposals in segmentation, reducing inference time and improving coverage without requiring full distribution knowledge.

Findings

Outperforms generative models in coverage and speed

Reduces inference time significantly

Applicable to real-world datasets without full distribution info

Abstract

Many segmentation tasks, such as medical image segmentation or future state prediction, are inherently ambiguous, meaning that multiple predictions are equally correct. Current methods typically rely on generative models to capture this uncertainty. However, identifying the underlying modes of the distribution with these methods is computationally expensive, requiring large numbers of samples and post-hoc clustering. In this paper, we shift the focus from stochastic sampling to the direct generation of likely outcomes. We introduce mode proposal models, a deterministic framework that efficiently produces a fixed-size set of proposal masks in a single forward pass. To handle superfluous proposals, we adapt a confidence mechanism, traditionally used in object detection, to the high-dimensional space of segmentation masks. Our approach significantly reduces inference time while achieving higher ground-truth coverage than existing generative models. Furthermore, we demonstrate that our model can be trained without knowing the full distribution of outcomes, making it applicable to real-world datasets. Finally, we show that by decomposing the velocity field of a pre-trained flow model, we can efficiently estimate prior mode probabilities for our proposals.