MedicoSAM: Robust Improvement of SAM for Medical Imaging

TL;DR

MedicoSAM enhances the Segment Anything model for medical imaging by applying finetuning strategies, significantly improving interactive segmentation performance and providing a practical, publicly available tool for medical image analysis.

Contribution

This work introduces MedicoSAM, a finetuned version of Segment Anything optimized for medical images, demonstrating improved interactive segmentation capabilities.

Findings

Improved performance in interactive segmentation tasks.

Semantic segmentation benefits are limited without medical-specific pretraining.

MedicoSAM is publicly available and compatible with existing annotation tools.

Abstract

Medical image segmentation is an important analysis task in clinical practice and research. Deep learning has massively advanced the field, but current approaches are mostly based on models trained for a specific task. Training such models or adapting them to a new condition is costly due to the need for (manually) labeled data. The emergence of vision foundation models, especially Segment Anything, offers a path to universal segmentation for medical images, overcoming these issues. Here, we study how to improve Segment Anything for medical images by comparing different finetuning strategies on a large and diverse dataset. We evaluate the finetuned models on a wide range of interactive and (automatic) semantic segmentation tasks. We find that the performance can be clearly improved for interactive segmentation. However, semantic segmentation does not benefit from pretraining on medical images. Our best model, MedicoSAM, is publicly available at https://github.com/computational-cell-analytics/medico-sam. We show that it is compatible with existing tools for data annotation and believe that it will be of great practical value.