MIAS-SAM: Medical Image Anomaly Segmentation without thresholding

TL;DR

MIAS-SAM introduces a threshold-free method for medical image anomaly segmentation using a patch-based memory bank and SAM encoder, achieving accurate results across multiple imaging modalities.

Contribution

The paper proposes a novel threshold-free anomaly segmentation approach leveraging a patch-based memory bank and SAM encoder, improving accuracy without threshold tuning.

Findings

Achieves high DICE scores on three diverse datasets

Does not require thresholding for segmentation

Effective across MRI, CT, and OCT modalities

Abstract

This paper presents MIAS-SAM, a novel approach for the segmentation of anomalous regions in medical images. MIAS-SAM uses a patch-based memory bank to store relevant image features, which are extracted from normal data using the SAM encoder. At inference time, the embedding patches extracted from the SAM encoder are compared with those in the memory bank to obtain the anomaly map. Finally, MIAS-SAM computes the center of gravity of the anomaly map to prompt the SAM decoder, obtaining an accurate segmentation from the previously extracted features. Differently from prior works, MIAS-SAM does not require to define a threshold value to obtain the segmentation from the anomaly map. Experimental results conducted on three publicly available datasets, each with a different imaging modality (Brain MRI, Liver CT, and Retina OCT) show accurate anomaly segmentation capabilities measured using DICE score. The code is available at: https://github.com/warpcut/MIAS-SAM