Is SAM3 ready for pathology segmentation?

TL;DR

This paper systematically evaluates SAM3's ability to segment pathology images across various supervision settings, revealing its limitations and guiding future domain adaptation efforts.

Contribution

It introduces a structured evaluation protocol for SAM3 in pathology segmentation and provides insights into its performance and limitations.

Findings

Text prompts poorly activate nuclear concepts.

Performance varies significantly with prompt types and budgets.

Few-shot learning improves results but lacks robustness.

Abstract

Is Segment Anything Model 3 (SAM3) capable in segmenting Any Pathology Images? Digital pathology segmentation spans tissue-level and nuclei-level scales, where traditional methods often suffer from high annotation costs and poor generalization. SAM3 introduces Promptable Concept Segmentation, offering a potential automated interface via text prompts. With this work, we propose a systematic evaluation protocol to explore the capability space of SAM3 in a structured manner. Specifically, we evaluate SAM3 under different supervision settings including zero-shot, few-shot, and supervised with varying prompting strategies. Our extensive evaluation on pathological datasets including NuInsSeg, PanNuke and GlaS, reveals that: (1) text-only prompts poorly activate nuclear concepts; (2) performance is highly sensitive to visual prompt types and budgets; (3) few-shot learning offers gains, but SAM3 lacks robustness against visual prompt noise; and (4) a significant gap persists between prompt-based usage and task-trained adapter-based reference. Our study delineates SAM3's boundaries in pathology image segmentation and provides practical guidance on the necessity of pathology domain adaptation.