Probabilistic Modeling of Multi-rater Medical Image Segmentation for Diversity and Personalization

TL;DR

ProSeg is a probabilistic model that generates diverse, personalized lesion segmentations by capturing expert preferences and boundary ambiguity through variational inference.

Contribution

It introduces a novel probabilistic framework with latent variables for simultaneous diversification and personalization in multi-rater medical image segmentation.

Findings

Achieves state-of-the-art performance on NPC and LIDC-IDRI datasets.

Produces segmentation outputs that are both diverse and tailored to individual experts.

Outperforms existing models constrained to either diversity or personalization.

Abstract

Lesion segmentation is inherently influenced by imaging uncertainty, arising from ill-defined lesion boundaries and inter-observer variability in diagnosis. To address this challenge, previous works formulated the multi-rater medical image segmentation task, where multiple experts provide separate annotations for each image. However, existing models are typically constrained to either generate diverse segmentation that lacks expert specificity or to produce personalized outputs that merely replicate individual annotators. We propose \textbf{Pro}babilistic modeling of multi-rater lesion \textbf{Seg}mentation (\textbf{ProSeg}) that simultaneously enables both diversification and personalization. Specifically, we introduce two latent variables to model expert annotation preferences and lesion boundary ambiguity. Their conditional probabilistic distributions are then obtained through variational inference, allowing segmentation outputs to be generated by sampling from these distributions. Extensive experiments on both the nasopharyngeal carcinoma dataset (NPC) and the lung nodule dataset (LIDC-IDRI) demonstrate that our ProSeg achieves a new state-of-the-art performance, providing segmentation results that are both diverse and expert-personalized.