Boosting SAM for Cross-Domain Few-Shot Segmentation via Conditional Point Sparsification

TL;DR

This paper introduces Conditional Point Sparsification (CPS), a training-free method that improves cross-domain few-shot segmentation by adaptively selecting key points for better SAM interactions, especially in challenging domains like medical and satellite imagery.

Contribution

The paper proposes CPS, a novel approach that enhances SAM's performance in cross-domain few-shot segmentation by adaptively sparsifying matched points without additional training.

Findings

CPS outperforms existing SAM-based methods on multiple CD-FSS datasets.

Point sparsification improves segmentation accuracy in cross-domain scenarios.

CPS effectively handles large domain shifts in medical and satellite images.

Abstract

Motivated by the success of the Segment Anything Model (SAM) in promptable segmentation, recent studies leverage SAM to develop training-free solutions for few-shot segmentation, which aims to predict object masks in the target image based on a few reference exemplars. These SAM-based methods typically rely on point matching between reference and target images and use the matched dense points as prompts for mask prediction. However, we observe that dense points perform poorly in Cross-Domain Few-Shot Segmentation (CD-FSS), where target images are from medical or satellite domains. We attribute this issue to large domain shifts that disrupt the point-image interactions learned by SAM, and find that point density plays a crucial role under such conditions. To address this challenge, we propose Conditional Point Sparsification (CPS), a training-free approach that adaptively guides SAM interactions for cross-domain images based on reference exemplars. Leveraging ground-truth masks, the reference images provide reliable guidance for adaptively sparsifying dense matched points, enabling more accurate segmentation results. Extensive experiments demonstrate that CPS outperforms existing training-free SAM-based methods across diverse CD-FSS datasets.