SAM-U: Multi-box prompts triggered uncertainty estimation for reliable SAM in medical image

TL;DR

This paper introduces SAM-U, a method that uses multi-box prompts and uncertainty estimation to enhance the reliability and performance of SAM in medical image segmentation, addressing concerns of trustworthiness in healthcare applications.

Contribution

It presents a novel multi-box prompts triggered uncertainty estimation framework for SAM, improving segmentation reliability in medical imaging.

Findings

Multi-box prompts augmentation improves SAM performance.

Uncertainty estimation per pixel enhances reliability.

First paradigm for reliable SAM in medical imaging.

Abstract

Recently, Segmenting Anything has taken an important step towards general artificial intelligence. At the same time, its reliability and fairness have also attracted great attention, especially in the field of health care. In this study, we propose multi-box prompts triggered uncertainty estimation for SAM cues to demonstrate the reliability of segmented lesions or tissues. We estimate the distribution of SAM predictions via Monte Carlo with prior distribution parameters, which employs different prompts as formulation of test-time augmentation. Our experimental results found that multi-box prompts augmentation improve the SAM performance, and endowed each pixel with uncertainty. This provides the first paradigm for a reliable SAM.