Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift

TL;DR

This paper introduces a generalized label shift assumption to enhance domain adaptation methods, providing theoretical guarantees and practical reweighting techniques that improve performance under label distribution mismatches.

Contribution

It proposes the generalized label shift assumption, offers theoretical transfer guarantees, and develops a reweighting method that improves existing domain adaptation algorithms.

Findings

Improved performance on standard DA tasks with label mismatches

Theoretical guarantees for transfer performance under GLS

Effective reweighting method with low computational overhead

Abstract

Adversarial learning has demonstrated good performance in the unsupervised domain adaptation setting, by learning domain-invariant representations. However, recent work has shown limitations of this approach when label distributions differ between the source and target domains. In this paper, we propose a new assumption, generalized label shift ($GLS$), to improve robustness against mismatched label distributions. $GLS$ states that, conditioned on the label, there exists a representation of the input that is invariant between the source and target domains. Under $GLS$, we provide theoretical guarantees on the transfer performance of any classifier. We also devise necessary and sufficient conditions for $GLS$ to hold, by using an estimation of the relative class weights between domains and an appropriate reweighting of samples. Our weight estimation method could be straightforwardly and generically applied in existing domain adaptation (DA) algorithms that learn domain-invariant representations, with small computational overhead. In particular, we modify three DA algorithms, JAN, DANN and CDAN, and evaluate their performance on standard and artificial DA tasks. Our algorithms outperform the base versions, with vast improvements for large label distribution mismatches. Our code is available at https://tinyurl.com/y585xt6j.