One Polyp Identifies All: One-Shot Polyp Segmentation with SAM via Cascaded Priors and Iterative Prompt Evolution

TL;DR

This paper introduces OP-SAM, a novel one-shot polyp segmentation framework utilizing SAM with automatic prompt generation, iterative refinement, and size adaptation, achieving superior accuracy across multiple datasets without additional annotations.

Contribution

The paper presents a new one-shot segmentation method that automatically generates prompts from a single image and iteratively refines segmentation, reducing annotation effort and improving generalization.

Findings

Achieves 76.93% IoU on Kvasir dataset, surpassing state-of-the-art by 11.44%.

Effective in handling polyp size variations and domain shifts.

Reduces need for extensive annotations in medical image segmentation.

Abstract

Polyp segmentation is vital for early colorectal cancer detection, yet traditional fully supervised methods struggle with morphological variability and domain shifts, requiring frequent retraining. Additionally, reliance on large-scale annotations is a major bottleneck due to the time-consuming and error-prone nature of polyp boundary labeling. Recently, vision foundation models like Segment Anything Model (SAM) have demonstrated strong generalizability and fine-grained boundary detection with sparse prompts, effectively addressing key polyp segmentation challenges. However, SAM's prompt-dependent nature limits automation in medical applications, since manually inputting prompts for each image is labor-intensive and time-consuming. We propose OP-SAM, a One-shot Polyp segmentation framework based on SAM that automatically generates prompts from a single annotated image, ensuring accurate and generalizable segmentation without additional annotation burdens. Our method introduces Correlation-based Prior Generation (CPG) for semantic label transfer and Scale-cascaded Prior Fusion (SPF) to adapt to polyp size variations as well as filter out noisy transfers. Instead of dumping all prompts at once, we devise Euclidean Prompt Evolution (EPE) for iterative prompt refinement, progressively enhancing segmentation quality. Extensive evaluations across five datasets validate OP-SAM's effectiveness. Notably, on Kvasir, it achieves 76.93% IoU, surpassing the state-of-the-art by 11.44%.