Zero-shot performance of the Segment Anything Model (SAM) in 2D medical imaging: A comprehensive evaluation and practical guidelines

TL;DR

This study evaluates the zero-shot capabilities of the Segment Anything Model (SAM) in diverse medical imaging modalities, demonstrating its competitive performance and providing practical guidelines for effective use with minimal interaction.

Contribution

It is the first comprehensive evaluation of SAM's zero-shot performance in medical imaging, offering practical guidelines for its application across multiple modalities.

Findings

SAM's zero-shot performance is comparable or superior to existing methods.

Eight prompt strategies were tested across six datasets from four modalities.

Practical guidelines enable robust results with minimal interaction.

Abstract

Segmentation in medical imaging is a critical component for the diagnosis, monitoring, and treatment of various diseases and medical conditions. Presently, the medical segmentation landscape is dominated by numerous specialized deep learning models, each fine-tuned for specific segmentation tasks and image modalities. The recently-introduced Segment Anything Model (SAM) employs the ViT neural architecture and harnesses a massive training dataset to segment nearly any object; however, its suitability to the medical domain has not yet been investigated. In this study, we explore the zero-shot performance of SAM in medical imaging by implementing eight distinct prompt strategies across six datasets from four imaging modalities, including X-ray, ultrasound, dermatoscopy, and colonoscopy. Our findings reveal that SAM's zero-shot performance is not only comparable to, but in certain cases, surpasses the current state-of-the-art. Based on these results, we propose practical guidelines that require minimal interaction while consistently yielding robust outcomes across all assessed contexts. The source code, along with a demonstration of the recommended guidelines, can be accessed at https://github.com/Malta-Lab/SAM-zero-shot-in-Medical-Imaging.