Zero-shot capability of SAM-family models for bone segmentation in CT scans

TL;DR

This study evaluates the zero-shot bone segmentation capabilities of SAM-family models in CT scans, analyzing how different prompting strategies affect performance across skeletal regions, and providing guidelines for optimal use.

Contribution

It offers a comprehensive evaluation of SAM models for bone CT segmentation using various prompting strategies, filling a gap in medical imaging assessment.

Findings

Bounding box plus center point prompts yield best results.

Model size and dataset characteristics influence performance.

Guidelines for effective 2D prompting in medical segmentation are provided.

Abstract

The Segment Anything Model (SAM) and similar models build a family of promptable foundation models (FMs) for image and video segmentation. The object of interest is identified using prompts, such as bounding boxes or points. With these FMs becoming part of medical image segmentation, extensive evaluation studies are required to assess their strengths and weaknesses in clinical setting. Since the performance is highly dependent on the chosen prompting strategy, it is important to investigate different prompting techniques to define optimal guidelines that ensure effective use in medical image segmentation. Currently, no dedicated evaluation studies exist specifically for bone segmentation in CT scans, leaving a gap in understanding the performance for this task. Thus, we use non-iterative, ``optimal'' prompting strategies composed of bounding box, points and combinations to test the zero-shot capability of SAM-family models for bone CT segmentation on three different skeletal regions. Our results show that the best settings depend on the model type and size, dataset characteristics and objective to optimize. Overall, SAM and SAM2 prompted with a bounding box in combination with the center point for all the components of an object yield the best results across all tested settings. As the results depend on multiple factors, we provide a guideline for informed decision-making in 2D prompting with non-interactive, ''optimal'' prompts.