False Negative/Positive Control for SAM on Noisy Medical Images

TL;DR

This paper introduces a prompt augmentation and correction technique to enhance SAM's segmentation accuracy in noisy medical images, along with a method for 3D segmentation from minimal annotations.

Contribution

It proposes a novel prompt augmentation and false-negative/positive correction strategy for SAM, and introduces SS2V for 3D segmentation from a single 2D slice.

Findings

Improved segmentation performance on ultrasound datasets

Enhanced robustness to prompt inaccuracies

Enables 3D segmentation from minimal annotations

Abstract

The Segment Anything Model (SAM) is a recently developed all-range foundation model for image segmentation. It can use sparse manual prompts such as bounding boxes to generate pixel-level segmentation in natural images but struggles in medical images such as low-contrast, noisy ultrasound images. We propose a refined test-phase prompt augmentation technique designed to improve SAM's performance in medical image segmentation. The method couples multi-box prompt augmentation and an aleatoric uncertainty-based false-negative (FN) and false-positive (FP) correction (FNPC) strategy. We evaluate the method on two ultrasound datasets and show improvement in SAM's performance and robustness to inaccurate prompts, without the necessity for further training or tuning. Moreover, we present the Single-Slice-to-Volume (SS2V) method, enabling 3D pixel-level segmentation using only the bounding box annotation from a single 2D slice. Our results allow efficient use of SAM in even noisy, low-contrast medical images. The source code will be released soon.