SAM & SAM 2 in 3D Slicer: SegmentWithSAM Extension for Annotating Medical Images

TL;DR

This paper introduces an extension for 3D Slicer that adapts the Segment Anything Model 2 (SAM 2) for efficient annotation of 3D medical images, enabling prompt-based segmentation with easy propagation across volumes.

Contribution

The authors adapt SAM 2 for 3D medical image annotation and integrate it into 3D Slicer as a user-friendly extension, facilitating faster and more accurate segmentation.

Findings

Extension enables point-based prompts for 3D medical images.

Annotations can be propagated across entire volumes.

Implementation is publicly available for easy use.

Abstract

Creating annotations for 3D medical data is time-consuming and often requires highly specialized expertise. Various tools have been implemented to aid this process. Segment Anything Model 2 (SAM 2) offers a general-purpose prompt-based segmentation algorithm designed to annotate videos. In this paper, we adapt this model to the annotation of 3D medical images and offer our implementation in the form of an extension to the popular annotation software: 3D Slicer. Our extension allows users to place point prompts on 2D slices to generate annotation masks and propagate these annotations across entire volumes in either single-directional or bi-directional manners. Our code is publicly available on https://github.com/mazurowski-lab/SlicerSegmentWithSAM and can be easily installed directly from the Extension Manager of 3D Slicer as well.