Intelligent Communication Mixture-of-Experts Boosted-Medical Image Segmentation Foundation Model

TL;DR

This paper introduces IC-MoE, a novel foundation model for medical image segmentation that enhances high-level feature representation and preserves pretrained weights through expert construction, adaptive voting, and contrastive learning, outperforming state-of-the-art methods.

Contribution

The paper proposes a new IC-MoE model with expert design, adaptive voting, and semantic-guided contrastive learning to improve medical image segmentation foundation models.

Findings

IC-MoE outperforms SOTA models on three public datasets.

Enhanced high-level feature representation and structural integrity.

Demonstrated superior generalizability across diverse scenarios.

Abstract

Foundation models for medical image segmentation have achieved remarkable performance. Adaptive fine-tuning of natural image segmentation foundation models is crucial for medical image segmentation tasks. However, some limitations exist in existing fine-tuning methods: 1) insufficient representation of high-level features and 2) the fine-tuning process disrupts the structural integrity of pretrained weights. Inspired by these critical problems, we propose an intelligent communication mixture-of-experts boosted-medical image segmentation foundation model, named IC-MoE, with twofold ideas: 1) We construct basic experts, semantic experts, and adaptive experts. Moreover, we implement a pixel probability adaptive voting strategy, which enables expert selection and fusion through label consistency and load balancing. This approach preliminarily enhances the representation capability of high-level features while preserving the structural integrity of pretrained weights. 2) We propose a semantic-guided contrastive learning method to address the issue of weak supervision in contrastive learning. This method further enhances the representation capability of high-level features while preserving the structural integrity of pretrained weights. Extensive experiments across three public medical image segmentation datasets demonstrate that the IC-MoE outperforms other SOTA models. Consequently, the proposed IC-MoE effectively supplements foundational medical image segmentation models with high-level features and pretrained structural integrity. We also validate the superior generalizability of the IC-MoE across diverse medical image segmentation scenarios.