Adapting Segment Anything Model to Melanoma Segmentation in Microscopy Slide Images

TL;DR

This paper introduces a novel melanoma segmentation method in microscopy images by combining an initial segmentation model with the Segment Anything Model, employing dynamic prompts and refinement techniques to improve accuracy and outperform existing methods.

Contribution

The paper presents a new approach that integrates a semantic segmentation model with SAM using dynamic prompting and refinement for high-precision melanoma segmentation in microscopy images.

Findings

Outperforms state-of-the-art melanoma segmentation methods.

Achieves 9.1% higher IoU than baseline models.

Demonstrates effective dynamic prompting strategy for high-resolution images.

Abstract

Melanoma segmentation in Whole Slide Images (WSIs) is useful for prognosis and the measurement of crucial prognostic factors such as Breslow depth and primary invasive tumor size. In this paper, we present a novel approach that uses the Segment Anything Model (SAM) for automatic melanoma segmentation in microscopy slide images. Our method employs an initial semantic segmentation model to generate preliminary segmentation masks that are then used to prompt SAM. We design a dynamic prompting strategy that uses a combination of centroid and grid prompts to achieve optimal coverage of the super high-resolution slide images while maintaining the quality of generated prompts. To optimize for invasive melanoma segmentation, we further refine the prompt generation process by implementing in-situ melanoma detection and low-confidence region filtering. We select Segformer as the initial segmentation model and EfficientSAM as the segment anything model for parameter-efficient fine-tuning. Our experimental results demonstrate that this approach not only surpasses other state-of-the-art melanoma segmentation methods but also significantly outperforms the baseline Segformer by 9.1% in terms of IoU.