Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning

TL;DR

This paper introduces SSM-SAM, a meta-learning framework that significantly improves few-shot medical image segmentation by enabling rapid adaptation and better attention mechanisms, outperforming existing methods on CT and MRI datasets.

Contribution

The paper proposes a novel meta-learning based framework with self-sampling and attention modules for fast, accurate few-shot medical image segmentation, addressing the limitations of general models like SAM.

Findings

Achieves over 10% DSC improvement on abdominal CT dataset.

Attains rapid adaptation in under 1 minute for new organs.

Outperforms state-of-the-art few-shot segmentation methods.

Abstract

While the Segment Anything Model (SAM) excels in semantic segmentation for general-purpose images, its performance significantly deteriorates when applied to medical images, primarily attributable to insufficient representation of medical images in its training dataset. Nonetheless, gathering comprehensive datasets and training models that are universally applicable is particularly challenging due to the long-tail problem common in medical images. To address this gap, here we present a Self-Sampling Meta SAM (SSM-SAM) framework for few-shot medical image segmentation. Our innovation lies in the design of three key modules: 1) An online fast gradient descent optimizer, further optimized by a meta-learner, which ensures swift and robust adaptation to new tasks. 2) A Self-Sampling module designed to provide well-aligned visual prompts for improved attention allocation; and 3) A robust attention-based decoder specifically designed for medical few-shot learning to capture relationship between different slices. Extensive experiments on a popular abdominal CT dataset and an MRI dataset demonstrate that the proposed method achieves significant improvements over state-of-the-art methods in few-shot segmentation, with an average improvements of 10.21% and 1.80% in terms of DSC, respectively. In conclusion, we present a novel approach for rapid online adaptation in interactive image segmentation, adapting to a new organ in just 0.83 minutes. Code is publicly available on GitHub upon acceptance.