Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation

TL;DR

This paper introduces a novel multi-modal MR brain tumor segmentation method that uses tumor prototypes and multi-expert feature integration to improve sub-region feature highlighting and segmentation accuracy.

Contribution

It proposes a prototype-driven feature representation and a mutual transmission mechanism to enhance single-modal features and address information aliasing issues.

Findings

Outperforms existing methods on three brain tumor segmentation datasets.

Effectively highlights tumor sub-region features using prototypes.

Improves segmentation accuracy through multi-expert feature fusion.

Abstract

For multi-modal magnetic resonance (MR) brain tumor image segmentation, current methods usually directly extract the discriminative features from input images for tumor sub-region category determination and localization. However, the impact of information aliasing caused by the mutual inclusion of tumor sub-regions is often ignored. Moreover, existing methods usually do not take tailored efforts to highlight the single tumor sub-region features. To this end, a multi-modal MR brain tumor segmentation method with tumor prototype-driven and multi-expert integration is proposed. It could highlight the features of each tumor sub-region under the guidance of tumor prototypes. Specifically, to obtain the prototypes with complete information, we propose a mutual transmission mechanism to transfer different modal features to each other to address the issues raised by insufficient information on single-modal features. Furthermore, we devise a prototype-driven feature representation and fusion method with the learned prototypes, which implants the prototypes into tumor features and generates corresponding activation maps. With the activation maps, the sub-region features consistent with the prototype category can be highlighted. A key information enhancement and fusion strategy with multi-expert integration is designed to further improve the segmentation performance. The strategy can integrate the features from different layers of the extra feature extraction network and the features highlighted by the prototypes. Experimental results on three competition brain tumor segmentation datasets prove the superiority of the proposed method.