Feature-enhanced Generation and Multi-modality Fusion based Deep Neural Network for Brain Tumor Segmentation with Missing MR Modalities

TL;DR

This paper introduces a novel deep neural network that effectively segments brain tumors from multimodal MRI data even when some modalities are missing, by generating missing modality features and leveraging inter-modality correlations.

Contribution

The work proposes a feature-enhanced generator and correlation constraint to synthesize missing MRI modalities, improving segmentation accuracy in incomplete multimodal data scenarios.

Findings

Achieves an average Dice Score of 82.9% for whole tumor.

Outperforms previous methods by up to 18.2%.

Effective in cases with missing MRI modalities.

Abstract

Using multimodal Magnetic Resonance Imaging (MRI) is necessary for accurate brain tumor segmentation. The main problem is that not all types of MRIs are always available in clinical exams. Based on the fact that there is a strong correlation between MR modalities of the same patient, in this work, we propose a novel brain tumor segmentation network in the case of missing one or more modalities. The proposed network consists of three sub-networks: a feature-enhanced generator, a correlation constraint block and a segmentation network. The feature-enhanced generator utilizes the available modalities to generate 3D feature-enhanced image representing the missing modality. The correlation constraint block can exploit the multi-source correlation between the modalities and also constrain the generator to synthesize a feature-enhanced modality which must have a coherent correlation with the available modalities. The segmentation network is a multi-encoder based U-Net to achieve the final brain tumor segmentation. The proposed method is evaluated on BraTS 2018 dataset. Experimental results demonstrate the effectiveness of the proposed method which achieves the average Dice Score of 82.9, 74.9 and 59.1 on whole tumor, tumor core and enhancing tumor, respectively across all the situations, and outperforms the best method by 3.5%, 17% and 18.2%.