MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignment

TL;DR

This paper introduces MedMAP, a novel alignment-based training paradigm for incomplete multi-modal brain tumor segmentation that effectively reduces modality gaps and improves performance without relying on pre-trained models.

Contribution

The paper proposes a new training paradigm that aligns latent features across modalities to a distribution anchor, ensuring invariant representations and theoretical effectiveness.

Findings

Improved segmentation accuracy on BraTS datasets

Models exhibit reduced modality gaps and invariant features

Theoretical proof of the paradigm's effectiveness

Abstract

Brain tumor segmentation is often based on multiple magnetic resonance imaging (MRI). However, in clinical practice, certain modalities of MRI may be missing, which presents a more difficult scenario. To cope with this challenge, Knowledge Distillation, Domain Adaption, and Shared Latent Space have emerged as commonly promising strategies. However, recent efforts typically overlook the modality gaps and thus fail to learn important invariant feature representations across different modalities. Such drawback consequently leads to limited performance for missing modality models. To ameliorate these problems, pre-trained models are used in natural visual segmentation tasks to minimize the gaps. However, promising pre-trained models are often unavailable in medical image segmentation tasks. Along this line, in this paper, we propose a novel paradigm that aligns latent features of involved modalities to a well-defined distribution anchor as the substitution of the pre-trained model}. As a major contribution, we prove that our novel training paradigm ensures a tight evidence lower bound, thus theoretically certifying its effectiveness. Extensive experiments on different backbones validate that the proposed paradigm can enable invariant feature representations and produce models with narrowed modality gaps. Models with our alignment paradigm show their superior performance on both BraTS2018 and BraTS2020 datasets.