$\mathrm{SAM^{Med}}$: A medical image annotation framework based on large vision model

TL;DR

This paper introduces $ ext{SAM}^{Med}$, a framework that enhances medical image annotation by leveraging the large vision model SAM, improving segmentation accuracy and accelerating annotation with minimal input points and automatic prompts.

Contribution

The paper presents $ ext{SAM}^{Med}$, a novel framework combining prompt-learning and automatic prompt generation to improve medical image segmentation with minimal annotations.

Findings

Significant segmentation accuracy improvement with only 5 input points.

Achieved Dice coefficients of 0.80 for kidney and 0.82 for liver segmentation.

Demonstrated the potential of large vision models in medical image annotation.

Abstract

Recently, large vision model, Segment Anything Model (SAM), has revolutionized the computer vision field, especially for image segmentation. SAM presented a new promptable segmentation paradigm that exhibit its remarkable zero-shot generalization ability. An extensive researches have explore the potential and limits of SAM in various downstream tasks. In this study, we presents $\mathrm{SAM^{Med}}$, an enhanced framework for medical image annotation that leverages the capabilities of SAM. $\mathrm{SAM^{Med}}$ framework consisted of two submodules, namely $\mathrm{SAM^{assist}}$ and $\mathrm{SAM^{auto}}$. The $\mathrm{SAM^{assist}}$ demonstrates the generalization ability of SAM to the downstream medical segmentation task using the prompt-learning approach. Results show a significant improvement in segmentation accuracy with only approximately 5 input points. The $\mathrm{SAM^{auto}}$ model aims to accelerate the annotation process by automatically generating input prompts. The proposed SAP-Net model achieves superior segmentation performance with only five annotated slices, achieving an average Dice coefficient of 0.80 and 0.82 for kidney and liver segmentation, respectively. Overall, $\mathrm{SAM^{Med}}$ demonstrates promising results in medical image annotation. These findings highlight the potential of leveraging large-scale vision models in medical image annotation tasks.