Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation

TL;DR

This paper introduces CPC-SAM, a semi-supervised medical image segmentation method leveraging the Segment Anything Model's prompt design and a cross-prompting strategy to improve learning from limited labeled data and abundant unlabeled data.

Contribution

It proposes a novel cross prompting consistency approach with SAM, including a dual-branch framework and prompt regularization, to enhance semi-supervised segmentation performance.

Findings

Achieves over 9% Dice improvement on breast cancer segmentation.

Outperforms state-of-the-art SSL methods across different data ratios and modalities.

Demonstrates effective learning from scarce labeled and abundant unlabeled data.

Abstract

Semi-supervised learning (SSL) has achieved notable progress in medical image segmentation. To achieve effective SSL, a model needs to be able to efficiently learn from limited labeled data and effectively exploiting knowledge from abundant unlabeled data. Recent developments in visual foundation models, such as the Segment Anything Model (SAM), have demonstrated remarkable adaptability with improved sample efficiency. To harness the power of foundation models for application in SSL, we propose a cross prompting consistency method with segment anything model (CPC-SAM) for semi-supervised medical image segmentation. Our method employs SAM's unique prompt design and innovates a cross-prompting strategy within a dual-branch framework to automatically generate prompts and supervisions across two decoder branches, enabling effectively learning from both scarce labeled and valuable unlabeled data. We further design a novel prompt consistency regularization, to reduce the prompt position sensitivity and to enhance the output invariance under different prompts. We validate our method on two medical image segmentation tasks. The extensive experiments with different labeled-data ratios and modalities demonstrate the superiority of our proposed method over the state-of-the-art SSL methods, with more than 9% Dice improvement on the breast cancer segmentation task.