SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation

TL;DR

This paper introduces SSS, a semi-supervised medical image segmentation method leveraging SAM-2's features, a novel feature enhancement mechanism, and a prompt generator with physical constraints, achieving state-of-the-art results.

Contribution

The paper proposes a novel semi-supervised segmentation framework using SAM-2, incorporating a discriminative feature enhancement and a prompt generator with physical constraints for improved medical image segmentation.

Findings

Achieves a 53.15 Dice score on BHSD dataset.

Surpasses previous state-of-the-art by +3.65 Dice.

Demonstrates effectiveness on multi-label medical datasets.

Abstract

In the era of information explosion, efficiently leveraging large-scale unlabeled data while minimizing the reliance on high-quality pixel-level annotations remains a critical challenge in the field of medical imaging. Semi-supervised learning (SSL) enhances the utilization of unlabeled data by facilitating knowledge transfer, significantly improving the performance of fully supervised models and emerging as a highly promising research direction in medical image analysis. Inspired by the ability of Vision Foundation Models (e.g., SAM-2) to provide rich prior knowledge, we propose SSS (Semi-Supervised SAM-2), a novel approach that leverages SAM-2's robust feature extraction capabilities to uncover latent knowledge in unlabeled medical images, thus effectively enhancing feature support for fully supervised medical image segmentation. Specifically, building upon the single-stream "weak-to-strong" consistency regularization framework, this paper introduces a Discriminative Feature Enhancement (DFE) mechanism to further explore the feature discrepancies introduced by various data augmentation strategies across multiple views. By leveraging feature similarity and dissimilarity across multi-scale augmentation techniques, the method reconstructs and models the features, thereby effectively optimizing the salient regions. Furthermore, a prompt generator is developed that integrates Physical Constraints with a Sliding Window (PCSW) mechanism to generate input prompts for unlabeled data, fulfilling SAM-2's requirement for additional prompts. Extensive experiments demonstrate the superiority of the proposed method for semi-supervised medical image segmentation on two multi-label datasets, i.e., ACDC and BHSD. Notably, SSS achieves an average Dice score of 53.15 on BHSD, surpassing the previous state-of-the-art method by +3.65 Dice. Code will be available at https://github.com/AIGeeksGroup/SSS.