SAM-Path: A Segment Anything Model for Semantic Segmentation in Digital Pathology

TL;DR

This paper adapts the Segment Anything Model for semantic segmentation in digital pathology by introducing trainable class prompts and a pathology encoder, significantly improving segmentation accuracy on public datasets.

Contribution

The authors propose SAM-Path, a novel framework that enhances SAM for semantic pathology segmentation through trainable prompts and a pathology foundation model, addressing prior limitations.

Findings

27.52% improvement in Dice score over vanilla SAM

71.63% improvement in IOU over vanilla SAM

Additional pathology foundation model yields 5%+ relative gains in Dice and IOU

Abstract

Semantic segmentations of pathological entities have crucial clinical value in computational pathology workflows. Foundation models, such as the Segment Anything Model (SAM), have been recently proposed for universal use in segmentation tasks. SAM shows remarkable promise in instance segmentation on natural images. However, the applicability of SAM to computational pathology tasks is limited due to the following factors: (1) lack of comprehensive pathology datasets used in SAM training and (2) the design of SAM is not inherently optimized for semantic segmentation tasks. In this work, we adapt SAM for semantic segmentation by introducing trainable class prompts, followed by further enhancements through the incorporation of a pathology encoder, specifically a pathology foundation model. Our framework, SAM-Path enhances SAM's ability to conduct semantic segmentation in digital pathology without human input prompts. Through experiments on two public pathology datasets, the BCSS and the CRAG datasets, we demonstrate that the fine-tuning with trainable class prompts outperforms vanilla SAM with manual prompts and post-processing by 27.52% in Dice score and 71.63% in IOU. On these two datasets, the proposed additional pathology foundation model further achieves a relative improvement of 5.07% to 5.12% in Dice score and 4.50% to 8.48% in IOU.