Harmonized Feature Conditioning and Frequency-Prompt Personalization for Multi-Rater Medical Segmentation

TL;DR

This paper introduces a probabilistic framework for multi-rater medical image segmentation that disentangles scanner artifacts from true annotation variability, enabling personalized, uncertainty-aware, and more accurate segmentations.

Contribution

It proposes a novel harmonized feature conditioning approach with frequency-domain prompts and a regularization method to better model expert disagreement and improve segmentation quality.

Findings

Achieves state-of-the-art aggregated and individualized segmentation performance.

Reduces generalized energy distance and improves Dice scores, especially on noisy cases.

Provides clinically meaningful uncertainty estimates aligned with expert disagreement.

Abstract

Multi-rater medical image segmentation captures the inherent ambiguity of clinical interpretation, where diagnostic boundaries vary across experts and imaging devices. Existing approaches often reduce this diversity to consensus labels or treat rater differences as noise, resulting in overconfident and poorly calibrated models. We propose a harmonized probabilistic framework that disentangles acquisition artifacts from genuine annotator variability through adaptive feature conditioning and frequency-domain personalization. A lightweight Harmonizer Network implicitly models scanner-specific artifacts and performs dynamic feature modulation to standardize latent representations, ensuring that uncertainty reflects anatomy rather than noise. To represent rater-specific styles, we introduce a novel High-Frequency Prompt Modules that operate in the spectral domain to encode annotator-dependent boundary precision and textural sensitivity. These prompts adaptively modulate harmonized features to produce personalized yet anatomically consistent segmentations. Furthermore, a Generalized Energy Distance based regularization aligns the generative distribution with empirical annotation variability, promoting diversity where experts disagree and consensus where they converge. Experiments on LIDC-IDRI and NPC-170 show SOTA aggregated and individualized segmentation, with notable GED reductions and improved Dice scores, especially on noisy cases. Beyond accuracy, the model exhibits clinically meaningful uncertainty. Confidence rises in agreement regions and declines in ambiguous areas, supporting its use as a reliable and interpretable tool for multi-expert clinical workflows.