SfMamba: Efficient Source-Free Domain Adaptation via Selective Scan Modeling

TL;DR

SfMamba is a novel source-free domain adaptation method that leverages selective scan modeling and channel-wise feature extraction to improve domain-invariant features efficiently, outperforming existing approaches across benchmarks.

Contribution

It introduces Channel-wise Visual State-Space blocks and Semantic-Consistent Shuffle strategies for enhanced domain adaptation without source data.

Findings

Outperforms existing SFDA methods on multiple benchmarks.

Maintains parameter efficiency while improving accuracy.

Effectively captures domain-invariant features under significant shifts.

Abstract

Source-free domain adaptation (SFDA) tackles the critical challenge of adapting source-pretrained models to unlabeled target domains without access to source data, overcoming data privacy and storage limitations in real-world applications. However, existing SFDA approaches struggle with the trade-off between perception field and computational efficiency in domain-invariant feature learning. Recently, Mamba has offered a promising solution through its selective scan mechanism, which enables long-range dependency modeling with linear complexity. However, the Visual Mamba (i.e., VMamba) remains limited in capturing channel-wise frequency characteristics critical for domain alignment and maintaining spatial robustness under significant domain shifts. To address these, we propose a framework called SfMamba to fully explore the stable dependency in source-free model transfer. SfMamba introduces Channel-wise Visual State-Space block that enables channel-sequence scanning for domain-invariant feature extraction. In addition, SfMamba involves a Semantic-Consistent Shuffle strategy that disrupts background patch sequences in 2D selective scan while preserving prediction consistency to mitigate error accumulation. Comprehensive evaluations across multiple benchmarks show that SfMamba achieves consistently stronger performance than existing methods while maintaining favorable parameter efficiency, offering a practical solution for SFDA. Our code is available at https://github.com/chenxi52/SfMamba.