DC-Seg: Disentangled Contrastive Learning for Brain Tumor Segmentation with Missing Modalities

TL;DR

DC-Seg introduces a disentangled contrastive learning approach that explicitly separates anatomical and modality-specific features, improving brain tumor segmentation accuracy especially when some modalities are missing, and demonstrates superior performance over existing methods.

Contribution

The paper proposes a novel disentangled contrastive learning framework that enhances robustness to missing modalities in brain segmentation tasks.

Findings

Outperforms state-of-the-art methods on BraTS 2020 dataset

Demonstrates strong generalizability on WMH segmentation

Effectively handles incomplete multimodal data

Abstract

Accurate segmentation of brain images typically requires the integration of complementary information from multiple image modalities. However, clinical data for all modalities may not be available for every patient, creating a significant challenge. To address this, previous studies encode multiple modalities into a shared latent space. While somewhat effective, it remains suboptimal, as each modality contains distinct and valuable information. In this study, we propose DC-Seg (Disentangled Contrastive Learning for Segmentation), a new method that explicitly disentangles images into modality-invariant anatomical representation and modality-specific representation, by using anatomical contrastive learning and modality contrastive learning respectively. This solution improves the separation of anatomical and modality-specific features by considering the modality gaps, leading to more robust representations. Furthermore, we introduce a segmentation-based regularizer that enhances the model's robustness to missing modalities. Extensive experiments on the BraTS 2020 and a private white matter hyperintensity(WMH) segmentation dataset demonstrate that DC-Seg outperforms state-of-the-art methods in handling incomplete multimodal brain tumor segmentation tasks with varying missing modalities, while also demonstrate strong generalizability in WMH segmentation. The code is available at https://github.com/CuCl-2/DC-Seg.