PGP-SAM: Prototype-Guided Prompt Learning for Efficient Few-Shot Medical Image Segmentation

TL;DR

PGP-SAM introduces a prototype-guided prompt learning method that efficiently adapts the Segment Anything Model for medical image segmentation using limited annotated samples, reducing manual prompt design.

Contribution

It proposes a novel prototype-based few-shot tuning approach with a contextual modulation and cross-attention mechanism for automatic prompt generation.

Findings

Achieves higher Dice scores than existing prompt-free SAM variants.

Uses only 10% of the 2D slices for training.

Demonstrates effectiveness on multi-organ and ventricle datasets.

Abstract

The Segment Anything Model (SAM) has demonstrated strong and versatile segmentation capabilities, along with intuitive prompt-based interactions. However, customizing SAM for medical image segmentation requires massive amounts of pixel-level annotations and precise point- or box-based prompt designs. To address these challenges, we introduce PGP-SAM, a novel prototype-based few-shot tuning approach that uses limited samples to replace tedious manual prompts. Our key idea is to leverage inter- and intra-class prototypes to capture class-specific knowledge and relationships. We propose two main components: (1) a plug-and-play contextual modulation module that integrates multi-scale information, and (2) a class-guided cross-attention mechanism that fuses prototypes and features for automatic prompt generation. Experiments on a public multi-organ dataset and a private ventricle dataset demonstrate that PGP-SAM achieves superior mean Dice scores compared with existing prompt-free SAM variants, while using only 10\% of the 2D slices.