A Bit More Bayesian: Domain-Invariant Learning with Uncertainty

TL;DR

This paper introduces a probabilistic Bayesian framework for domain-invariant learning that effectively handles domain shift and uncertainty, achieving state-of-the-art results in cross-domain visual recognition.

Contribution

It develops a novel Bayesian neural network approach that jointly learns domain-invariant representations and classifiers, addressing domain shift and uncertainty in a principled manner.

Findings

Achieves state-of-the-art accuracy on four benchmarks.

Demonstrates the benefits of Bayesian treatment in domain generalization.

Validates the effectiveness through ablation studies.

Abstract

Domain generalization is challenging due to the domain shift and the uncertainty caused by the inaccessibility of target domain data. In this paper, we address both challenges with a probabilistic framework based on variational Bayesian inference, by incorporating uncertainty into neural network weights. We couple domain invariance in a probabilistic formula with the variational Bayesian inference. This enables us to explore domain-invariant learning in a principled way. Specifically, we derive domain-invariant representations and classifiers, which are jointly established in a two-layer Bayesian neural network. We empirically demonstrate the effectiveness of our proposal on four widely used cross-domain visual recognition benchmarks. Ablation studies validate the synergistic benefits of our Bayesian treatment when jointly learning domain-invariant representations and classifiers for domain generalization. Further, our method consistently delivers state-of-the-art mean accuracy on all benchmarks.