Proxy Denoising for Source-Free Domain Adaptation
Song Tang, Wenxin Su, Yan Gan, Mao Ye, Jianwei Zhang, and Xiatian Zhu
TL;DR
This paper introduces Proxy Denoising (ProDe), a novel method for Source-Free Domain Adaptation that uses a proxy model to correct noisy predictions from Vision-Language models, improving adaptation across various challenging settings.
Contribution
ProDe leverages a proxy denoising mechanism and mutual knowledge distillation to enhance SFDA by addressing noise in Vision-Language model supervision during adaptation.
Findings
ProDe outperforms state-of-the-art methods in multiple SFDA settings.
The proxy denoising approach effectively reduces supervision noise.
ProDe demonstrates robustness in open set, partial set, and multi-source scenarios.
Abstract
Source-Free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to an unlabeled target domain with no access to the source data. Inspired by the success of large Vision-Language (ViL) models in many applications, the latest research has validated ViL's benefit for SFDA by using their predictions as pseudo supervision. However, we observe that ViL's supervision could be noisy and inaccurate at an unknown rate, introducing additional negative effects during adaption. To address this thus-far ignored challenge, we introduce a novel Proxy Denoising (ProDe) approach. The key idea is to leverage the ViL model as a proxy to facilitate the adaptation process towards the latent domain-invariant space. We design a proxy denoising mechanism to correct ViL's predictions, grounded on a proxy confidence theory that models the dynamic effect of proxy's divergence against the…
Peer Reviews
Decision·ICLR 2025 Oral
- **Presentation-** The paper is written well overall and conveys the central ideas of the work quite effectively. The paper is easy to follow and understand. Additionally, the authors have presented experiments on a wide range of benchmarks and settings. - **Novelty-** The authors claim that the paper is the first to analyze the inaccurate predictions of the teacher VLM in the context of SFDA and propose a method to alleviate the same. - **Results-** The authors present extensive experiments
### (a) Concerns with Proxy Confidence Theory - Theorem 1 provides a relation between the confidence of the VLM’s predictions and the confidence of the source model and the current training model. This is based on the approximation of the VLM’s predictions to a Gaussian distribution and further expressing this in terms of the confidence of the VLM’s predictions. - However, the intuition behind how a Gaussian distribution is considered for the VLM’s predictions is not entirely clear. Moreover, th
The paper is based on proxy confidence theory and designs a reliable denoising algorithm to reduce the prediction noise of ViL, addressing an important issue that has been neglected in the use of ViL for pseudo-supervision. This may facilitate subsequent related work.
Although the ViL model is obtained based on a large dataset, for specific source and target domains, the ViL model can approximate the domain-invariant space. The validity of this assumption requires further theoretical support.
The paper effectively addresses the important problem of source-free domain adaptation (SFDA), where models must adapt to new target domains without access to labeled source data—an increasingly relevant setup in practical scenarios where source data may be proprietary or sensitive. Demonstrating state-of-the-art performance on standard SFDA benchmarks, the proposed method showcases its robustness and potential impact in the field. The authors employ mutual knowledge distillation to synchronize
### Ambiguity and Misleading Terminology in Domain Invariance Claims: The authors claim that their method moves toward a “domain invariant space” $D_v$ starting from $D_t$ , but this terminology is misleading and theoretically problematic. If a domain-invariant space were achievable, there would be no need for adaptation across other domains, as all target domains would align seamlessly with this invariant space. However, the results suggest that domain shifts still impact the model and hence
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Multimodal Machine Learning Applications
MethodsSparse Evolutionary Training