EP-SAM: Weakly Supervised Histopathology Segmentation via Enhanced Prompt with Segment Anything

TL;DR

This paper introduces EP-SAM, a weakly supervised segmentation method for histopathology images that leverages the Segment Anything Model and class activation maps to reduce labeling costs and improve generalization.

Contribution

It presents a novel weakly supervised segmentation approach combining SAM-based pseudo-labeling with class activation maps for histopathology analysis.

Findings

Outperforms existing WSSS methods on breast cancer datasets

Achieves high accuracy with only 12GB GPU memory

Demonstrates effective generalization across multiple datasets

Abstract

This work proposes a novel approach beyond supervised learning for effective pathological image analysis, addressing the challenge of limited robust labeled data. Pathological diagnosis of diseases like cancer has conventionally relied on the evaluation of morphological features by physicians and pathologists. However, recent advancements in compute-aided diagnosis (CAD) systems are gaining significant attention as diagnostic support tools. Although the advancement of deep learning has improved CAD significantly, segmentation models typically require large pixel-level annotated dataset, and such labeling is expensive. Existing studies not based on supervised approaches still struggle with limited generalization, and no practical approach has emerged yet. To address this issue, we present a weakly supervised semantic segmentation (WSSS) model by combining class activation map and Segment Anything Model (SAM)-based pseudo-labeling. For effective pretraining, we adopt the SAM-a foundation model that is pretrained on large datasets and operates in zero-shot configurations using only coarse prompts. The proposed approach transfer enhanced Attention Dropout Layer's knowledge to SAM, thereby generating pseudo-labels. To demonstrate the superiority of the proposed method, experimental studies are conducted on histopathological breast cancer datasets. The proposed method outperformed other WSSS methods across three datasets, demonstrating its efficiency by achieving this with only 12GB of GPU memory during training. Our code is available at : https://github.com/QI-NemoSong/EP-SAM