Diffusion-Driven Progressive Target Manipulation for Source-Free Domain Adaptation

TL;DR

This paper introduces DPTM, a diffusion-based framework for source-free domain adaptation that progressively refines pseudo-target data, significantly improving performance in large domain discrepancy scenarios.

Contribution

The paper proposes a novel diffusion-driven method that transforms and refines pseudo-target domains, overcoming limitations of existing SFDA approaches.

Findings

Outperforms existing SFDA methods on four benchmarks.

Achieves up to 18.6% performance improvement in large domain gap scenarios.

Demonstrates state-of-the-art results across diverse datasets.

Abstract

Source-free domain adaptation (SFDA) is a challenging task that tackles domain shifts using only a pre-trained source model and unlabeled target data. Existing SFDA methods are restricted by the fundamental limitation of source-target domain discrepancy. Non-generation SFDA methods suffer from unreliable pseudo-labels in challenging scenarios with large domain discrepancies, while generation-based SFDA methods are evidently degraded due to enlarged domain discrepancies in creating pseudo-source data. To address this limitation, we propose a novel generation-based framework named Diffusion-Driven Progressive Target Manipulation (DPTM) that leverages unlabeled target data as references to reliably generate and progressively refine a pseudo-target domain for SFDA. Specifically, we divide the target samples into a trust set and a non-trust set based on the reliability of pseudo-labels to sufficiently and reliably exploit their information. For samples from the non-trust set, we develop a manipulation strategy to semantically transform them into the newly assigned categories, while simultaneously maintaining them in the target distribution via a latent diffusion model. Furthermore, we design a progressive refinement mechanism that progressively reduces the domain discrepancy between the pseudo-target domain and the real target domain via iterative refinement. Experimental results demonstrate that DPTM outperforms existing methods by a large margin and achieves state-of-the-art performance on four prevailing SFDA benchmark datasets with different scales. Remarkably, DPTM can significantly enhance the performance by up to 18.6% in scenarios with large source-target gaps.