MedSAM-U: Uncertainty-Guided Auto Multi-Prompt Adaptation for Reliable MedSAM

TL;DR

MedSAM-U introduces an uncertainty-guided framework that automatically refines prompts for medical image segmentation, significantly improving accuracy across multiple modalities by adapting to prompt uncertainties.

Contribution

This work presents MedSAM-U, a novel uncertainty-guided prompt adaptation method that enhances MedSAM's reliability and accuracy in medical image segmentation tasks.

Findings

Achieves 1.7% to 20.5% performance improvement over MedSAM.

Validates effectiveness across five diverse medical imaging datasets.

Demonstrates robustness to prompt variations and uncertainties.

Abstract

The Medical Segment Anything Model (MedSAM) has shown remarkable performance in medical image segmentation, drawing significant attention in the field. However, its sensitivity to varying prompt types and locations poses challenges. This paper addresses these challenges by focusing on the development of reliable prompts that enhance MedSAM's accuracy. We introduce MedSAM-U, an uncertainty-guided framework designed to automatically refine multi-prompt inputs for more reliable and precise medical image segmentation. Specifically, we first train a Multi-Prompt Adapter integrated with MedSAM, creating MPA-MedSAM, to adapt to diverse multi-prompt inputs. We then employ uncertainty-guided multi-prompt to effectively estimate the uncertainties associated with the prompts and their initial segmentation results. In particular, a novel uncertainty-guided prompts adaptation technique is then applied automatically to derive reliable prompts and their corresponding segmentation outcomes. We validate MedSAM-U using datasets from multiple modalities to train a universal image segmentation model. Compared to MedSAM, experimental results on five distinct modal datasets demonstrate that the proposed MedSAM-U achieves an average performance improvement of 1.7\% to 20.5\% across uncertainty-guided prompts.