Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference

TL;DR

This paper introduces a variational Bayesian inference method for domain generalization that effectively handles conditional and label shifts, improving cross-domain accuracy over traditional invariant feature learning methods.

Contribution

It proposes a novel Bayesian framework that aligns conditional distributions in latent space, addressing label shifts neglected by existing methods.

Findings

Robustness to label shift demonstrated in experiments.

Significant improvement in cross-domain accuracy.

Outperforms conventional invariant feature learning methods.

Abstract

In this work, we propose a domain generalization (DG) approach to learn on several labeled source domains and transfer knowledge to a target domain that is inaccessible in training. Considering the inherent conditional and label shifts, we would expect the alignment of $p(x|y)$ and $p(y)$. However, the widely used domain invariant feature learning (IFL) methods relies on aligning the marginal concept shift w.r.t. $p(x)$, which rests on an unrealistic assumption that $p(y)$ is invariant across domains. We thereby propose a novel variational Bayesian inference framework to enforce the conditional distribution alignment w.r.t. $p(x|y)$ via the prior distribution matching in a latent space, which also takes the marginal label shift w.r.t. $p(y)$ into consideration with the posterior alignment. Extensive experiments on various benchmarks demonstrate that our framework is robust to the label shift and the cross-domain accuracy is significantly improved, thereby achieving superior performance over the conventional IFL counterparts.