SAM-DA: Decoder Adapter for Efficient Medical Domain Adaptation

TL;DR

This paper introduces SAM-DA, a decoder adapter that enables efficient medical domain adaptation for semantic segmentation, achieving high performance with minimal additional training parameters.

Contribution

The paper presents a novel SAM adapter placed in the mask decoder that significantly reduces training parameters while maintaining competitive segmentation performance in medical imaging.

Findings

Outperforms existing methods on four datasets.

Uses less than 1% of SAM's total parameters for training.

Effective in both supervised and test-time domain adaptation.

Abstract

This paper addresses the domain adaptation challenge for semantic segmentation in medical imaging. Despite the impressive performance of recent foundational segmentation models like SAM on natural images, they struggle with medical domain images. Beyond this, recent approaches that perform end-to-end fine-tuning of models are simply not computationally tractable. To address this, we propose a novel SAM adapter approach that minimizes the number of trainable parameters while achieving comparable performances to full fine-tuning. The proposed SAM adapter is strategically placed in the mask decoder, offering excellent and broad generalization capabilities and improved segmentation across both fully supervised and test-time domain adaptation tasks. Extensive validation on four datasets showcases the adapter's efficacy, outperforming existing methods while training less than 1% of SAM's total parameters.