Semi-Supervised Multi-Modal Medical Image Segmentation for Complex Situations

TL;DR

This paper introduces a semi-supervised multi-modal medical image segmentation method that leverages multi-modal fusion and contrastive mutual learning to improve accuracy and robustness in complex scenarios with limited labeled data.

Contribution

It proposes a novel multi-stage fusion and enhancement strategy combined with contrastive mutual learning to effectively utilize unlabeled multi-modal medical images.

Findings

Superior performance on two multi-modal datasets

Enhanced robustness in complex segmentation scenarios

Effective leveraging of unlabeled data

Abstract

Semi-supervised learning addresses the issue of limited annotations in medical images effectively, but its performance is often inadequate for complex backgrounds and challenging tasks. Multi-modal fusion methods can significantly improve the accuracy of medical image segmentation by providing complementary information. However, they face challenges in achieving significant improvements under semi-supervised conditions due to the challenge of effectively leveraging unlabeled data. There is a significant need to create an effective and reliable multi-modal learning strategy for leveraging unlabeled data in semi-supervised segmentation. To address these issues, we propose a novel semi-supervised multi-modal medical image segmentation approach, which leverages complementary multi-modal information to enhance performance with limited labeled data. Our approach employs a multi-stage multi-modal fusion and enhancement strategy to fully utilize complementary multi-modal information, while reducing feature discrepancies and enhancing feature sharing and alignment. Furthermore, we effectively introduce contrastive mutual learning to constrain prediction consistency across modalities, thereby facilitating the robustness of segmentation results in semi-supervised tasks. Experimental results on two multi-modal datasets demonstrate the superior performance and robustness of the proposed framework, establishing its valuable potential for solving medical image segmentation tasks in complex scenarios.