Mitigating Negative Transfer via Reducing Environmental Disagreement

TL;DR

This paper introduces RED, a method that reduces environmental disagreement by disentangling causal and non-causal features, effectively mitigating negative transfer in unsupervised domain adaptation and improving performance.

Contribution

It proposes a causally disentangled learning approach with adversarial training to reduce environmental disagreement, addressing negative transfer in UDA.

Findings

RED outperforms existing methods in mitigating negative transfer

Experimental results show state-of-the-art performance

Disentangling features improves domain adaptation effectiveness

Abstract

Unsupervised Domain Adaptation~(UDA) focuses on transferring knowledge from a labeled source domain to an unlabeled target domain, addressing the challenge of \emph{domain shift}. Significant domain shifts hinder effective knowledge transfer, leading to \emph{negative transfer} and deteriorating model performance. Therefore, mitigating negative transfer is essential. This study revisits negative transfer through the lens of causally disentangled learning, emphasizing cross-domain discriminative disagreement on non-causal environmental features as a critical factor. Our theoretical analysis reveals that overreliance on non-causal environmental features as the environment evolves can cause discriminative disagreements~(termed \emph{environmental disagreement}), thereby resulting in negative transfer. To address this, we propose Reducing Environmental Disagreement~(RED), which disentangles each sample into domain-invariant causal features and domain-specific non-causal environmental features via adversarially training domain-specific environmental feature extractors in the opposite domains. Subsequently, RED estimates and reduces environmental disagreement based on domain-specific non-causal environmental features. Experimental results confirm that RED effectively mitigates negative transfer and achieves state-of-the-art performance.