SAM3D: Zero-Shot Semi-Automatic Segmentation in 3D Medical Images with the Segment Anything Model

TL;DR

SAM3D offers a fast, accurate, zero-shot semi-automatic segmentation method for 3D medical images, leveraging the Segment Anything Model with a four-step process that requires minimal manual input and no additional training.

Contribution

The paper introduces SAM3D, a novel zero-shot segmentation approach for 3D medical images that combines user prompts, volume slicing, and inference without model training or fine-tuning.

Findings

Achieves good segmentation performance across multiple imaging modalities.

Requires no additional training or fine-tuning of the model.

Reduces manual input significantly for 3D segmentation tasks.

Abstract

We introduce SAM3D, a new approach to semi-automatic zero-shot segmentation of 3D images building on the existing Segment Anything Model. We achieve fast and accurate segmentations in 3D images with a four-step strategy involving: user prompting with 3D polylines, volume slicing along multiple axes, slice-wide inference with a pretrained model, and recomposition and refinement in 3D. We evaluated SAM3D performance qualitatively on an array of imaging modalities and anatomical structures and quantify performance for specific structures in abdominal pelvic CT and brain MRI. Notably, our method achieves good performance with zero model training or finetuning, making it particularly useful for tasks with a scarcity of preexisting labeled data. By enabling users to create 3D segmentations of unseen data quickly and with dramatically reduced manual input, these methods have the potential to aid surgical planning and education, diagnostic imaging, and scientific research.