Disentangling Prompt Dependence to Evaluate Segmentation Reliability in Gynecological MRI

TL;DR

This paper introduces a framework to evaluate the robustness of promptable segmentation models in gynecological MRI by disentangling prompt ambiguity from local sensitivity, revealing their impact on segmentation reliability.

Contribution

It presents the first explicit formulation of prompt dependence that separates prompt ambiguity from local sensitivity, enabling interpretable assessment of segmentation robustness.

Findings

Strong negative correlation between metrics and segmentation performance

Low mutual correlation between the two proposed metrics

Metrics effectively identify prompt-related failure modes

Abstract

Promptable segmentation models (e.g., the Segment Anything Models) enable generalizable, zero-shot segmentation across diverse domains. Although predictions are deterministic for a fixed image-prompt pair, the robustness of these models to variations in user prompts, referred to as prompt dependence, remains underexplored. In safety-critical workflows with substantial inter-user variability, interpretable and informative frameworks are needed to evaluate prompt dependence. In this work, we assess the reliability of promptable segmentation by analyzing and measuring its sensitivity to prompt variability. We introduce the first formulation of prompt dependence that explicitly disentangles prompt ambiguity (inter-user variability) from local sensitivity (interaction imprecision), offering an interpretable view of segmentation robustness. Experiments on two female pelvic MRI datasets for uterus and bladder segmentation reveal a strong negative correlation between both metrics and segmentation performance, highlighting the value of our framework for assessing robustness. The two metrics have low mutual correlation, supporting the disentangled design of our formulation, and provide meaningful indicators of prompt-related failure modes.