SAMSA: Segment Anything Model Enhanced with Spectral Angles for Hyperspectral Interactive Medical Image Segmentation

TL;DR

SAMSA is an innovative interactive segmentation framework that combines RGB foundation models with spectral analysis to improve hyperspectral medical image segmentation, demonstrating high accuracy with minimal user input.

Contribution

The paper introduces SAMSA, a spectral feature fusion strategy that enhances hyperspectral segmentation, independent of spectral band count and resolution, enabling effective few-shot and zero-shot learning.

Findings

Achieved over 81% DICE with 1-click in medical datasets

Demonstrated effectiveness in few-shot and zero-shot scenarios

Flexible integration of datasets with different spectral characteristics

Abstract

Hyperspectral imaging (HSI) provides rich spectral information for medical imaging, yet encounters significant challenges due to data limitations and hardware variations. We introduce SAMSA, a novel interactive segmentation framework that combines an RGB foundation model with spectral analysis. SAMSA efficiently utilizes user clicks to guide both RGB segmentation and spectral similarity computations. The method addresses key limitations in HSI segmentation through a unique spectral feature fusion strategy that operates independently of spectral band count and resolution. Performance evaluation on publicly available datasets has shown 81.0% 1-click and 93.4% 5-click DICE on a neurosurgical and 81.1% 1-click and 89.2% 5-click DICE on an intraoperative porcine hyperspectral dataset. Experimental results demonstrate SAMSA's effectiveness in few-shot and zero-shot learning scenarios and using minimal training examples. Our approach enables seamless integration of datasets with different spectral characteristics, providing a flexible framework for hyperspectral medical image analysis.