3D Medical Multi-modal Segmentation Network Guided by Multi-source Correlation Constraint

TL;DR

This paper introduces a multi-modal segmentation network that leverages a correlation constraint to learn and utilize the strong relationships between different imaging modalities, improving brain tumor segmentation accuracy.

Contribution

The novel contribution is the integration of a linear correlation constraint and dual attention-based feature fusion within a multi-encoder segmentation network.

Findings

Improved segmentation accuracy on BraTS-2018 dataset.

Effective learning of correlated features across modalities.

Enhanced feature selection via dual attention mechanism.

Abstract

In the field of multimodal segmentation, the correlation between different modalities can be considered for improving the segmentation results. In this paper, we propose a multi-modality segmentation network with a correlation constraint. Our network includes N model-independent encoding paths with N image sources, a correlation constraint block, a feature fusion block, and a decoding path. The model independent encoding path can capture modality-specific features from the N modalities. Since there exists a strong correlation between different modalities, we first propose a linear correlation block to learn the correlation between modalities, then a loss function is used to guide the network to learn the correlated features based on the linear correlation block. This block forces the network to learn the latent correlated features which are more relevant for segmentation. Considering that not all the features extracted from the encoders are useful for segmentation, we propose to use dual attention based fusion block to recalibrate the features along the modality and spatial paths, which can suppress less informative features and emphasize the useful ones. The fused feature representation is finally projected by the decoder to obtain the segmentation result. Our experiment results tested on BraTS-2018 dataset for brain tumor segmentation demonstrate the effectiveness of our proposed method.