MMIS-Net for Retinal Fluid Segmentation and Detection

TL;DR

MMIS-Net is a novel multi-modal medical image segmentation network that leverages multiple datasets and similarity fusion to improve retinal fluid detection and segmentation accuracy, outperforming existing models.

Contribution

The paper introduces MMIS-Net with Similarity Fusion blocks and a one-hot label space, enabling effective multi-dataset training for improved medical image segmentation.

Findings

Achieved a mean Dice score of 0.83 on retinal fluid segmentation.

Attained a perfect AUC of 1 for fluid detection.

Outperformed state-of-the-art models on the RETOUCH challenge.

Abstract

Purpose: Deep learning methods have shown promising results in the segmentation, and detection of diseases in medical images. However, most methods are trained and tested on data from a single source, modality, organ, or disease type, overlooking the combined potential of other available annotated data. Numerous small annotated medical image datasets from various modalities, organs, and diseases are publicly available. In this work, we aim to leverage the synergistic potential of these datasets to improve performance on unseen data. Approach: To this end, we propose a novel algorithm called MMIS-Net (MultiModal Medical Image Segmentation Network), which features Similarity Fusion blocks that utilize supervision and pixel-wise similarity knowledge selection for feature map fusion. Additionally, to address inconsistent class definitions and label contradictions, we created a one-hot label space to handle classes absent in one dataset but annotated in another. MMIS-Net was trained on 10 datasets encompassing 19 organs across 2 modalities to build a single model. Results: The algorithm was evaluated on the RETOUCH grand challenge hidden test set, outperforming large foundation models for medical image segmentation and other state-of-the-art algorithms. We achieved the best mean Dice score of 0.83 and an absolute volume difference of 0.035 for the fluids segmentation task, as well as a perfect Area Under the Curve of 1 for the fluid detection task. Conclusion: The quantitative results highlight the effectiveness of our proposed model due to the incorporation of Similarity Fusion blocks into the network's backbone for supervision and similarity knowledge selection, and the use of a one-hot label space to address label class inconsistencies and contradictions.