DiffOSeg: Omni Medical Image Segmentation via Multi-Expert Collaboration Diffusion Model

TL;DR

DiffOSeg introduces a diffusion-based framework for medical image segmentation that captures both consensus and individual expert preferences, addressing annotation variability and improving performance over existing methods.

Contribution

The paper presents a novel two-stage diffusion model that simultaneously models consensus and individual expert preferences in medical image segmentation.

Findings

Outperforms state-of-the-art methods on LIDC-IDRI and NPC-170 datasets.

Effectively captures both consensus and expert-specific segmentation preferences.

Demonstrates robustness across multiple evaluation metrics.

Abstract

Annotation variability remains a substantial challenge in medical image segmentation, stemming from ambiguous imaging boundaries and diverse clinical expertise. Traditional deep learning methods producing single deterministic segmentation predictions often fail to capture these annotator biases. Although recent studies have explored multi-rater segmentation, existing methods typically focus on a single perspective -- either generating a probabilistic ``gold standard'' consensus or preserving expert-specific preferences -- thus struggling to provide a more omni view. In this study, we propose DiffOSeg, a two-stage diffusion-based framework, which aims to simultaneously achieve both consensus-driven (combining all experts' opinions) and preference-driven (reflecting experts' individual assessments) segmentation. Stage I establishes population consensus through a probabilistic consensus strategy, while Stage II captures expert-specific preference via adaptive prompts. Demonstrated on two public datasets (LIDC-IDRI and NPC-170), our model outperforms existing state-of-the-art methods across all evaluated metrics. Source code is available at https://github.com/string-ellipses/DiffOSeg .