S2S2: Semantic Stacking for Robust Semantic Segmentation in Medical Imaging

TL;DR

This paper introduces 'semantic stacking', a domain-agnostic, data-driven method that enhances medical image segmentation robustness by estimating denoised semantic representations, improving performance across diverse conditions without relying on domain-specific knowledge.

Contribution

The paper proposes a novel semantic stacking approach that is domain-agnostic and data-driven, improving segmentation robustness without requiring domain-specific assumptions.

Findings

Outperforms conventional methods in diverse conditions

Effective across various image modalities and architectures

Enhances segmentation accuracy and robustness

Abstract

Robustness and generalizability in medical image segmentation are often hindered by scarcity and limited diversity of training data, which stands in contrast to the variability encountered during inference. While conventional strategies -- such as domain-specific augmentation, specialized architectures, and tailored training procedures -- can alleviate these issues, they depend on the availability and reliability of domain knowledge. When such knowledge is unavailable, misleading, or improperly applied, performance may deteriorate. In response, we introduce a novel, domain-agnostic, add-on, and data-driven strategy inspired by image stacking in image denoising. Termed ``semantic stacking,'' our method estimates a denoised semantic representation that complements the conventional segmentation loss during training. This method does not depend on domain-specific assumptions, making it broadly applicable across diverse image modalities, model architectures, and augmentation techniques. Through extensive experiments, we validate the superiority of our approach in improving segmentation performance under diverse conditions. Code is available at https://github.com/ymp5078/Semantic-Stacking.