Adapting Foundation Models for Few-Shot Medical Image Segmentation: Actively and Sequentially

TL;DR

This paper introduces ASAP, a novel active and sequential domain adaptation framework for few-shot medical image segmentation, which dynamically selects auxiliary datasets to improve model performance across diverse medical imaging tasks.

Contribution

The paper proposes a new method that formulates auxiliary dataset selection as a multi-armed bandit problem, enabling adaptive and effective domain adaptation in few-shot medical image segmentation.

Findings

Achieves an average of 27.75% Dice score improvement on MRI datasets.

Achieves an average of 7.52% Dice score improvement on CT datasets.

Outperforms state-of-the-art FSDA methods significantly.

Abstract

Recent advances in foundation models have brought promising results in computer vision, including medical image segmentation. Fine-tuning foundation models on specific low-resource medical tasks has become a standard practice. However, ensuring reliable and robust model adaptation when the target task has a large domain gap and few annotated samples remains a challenge. Previous few-shot domain adaptation (FSDA) methods seek to bridge the distribution gap between source and target domains by utilizing auxiliary data. The selection and scheduling of auxiliaries are often based on heuristics, which can easily cause negative transfer. In this work, we propose an Active and Sequential domain AdaPtation (ASAP) framework for dynamic auxiliary dataset selection in FSDA. We formulate FSDA as a multi-armed bandit problem and derive an efficient reward function to prioritize training on auxiliary datasets that align closely with the target task, through a single-round fine-tuning. Empirical validation on diverse medical segmentation datasets demonstrates that our method achieves favorable segmentation performance, significantly outperforming the state-of-the-art FSDA methods, achieving an average gain of 27.75% on MRI and 7.52% on CT datasets in Dice score. Code is available at the git repository: https://github.com/techicoco/ASAP.