Domain Adaptation as a Problem of Inference on Graphical Models

TL;DR

This paper introduces a novel framework for unsupervised domain adaptation using graphical models to encode and infer the invariant and changing components of data distributions across domains, improving adaptation performance.

Contribution

It proposes a graphical model-based approach to automate domain adaptation by encoding distribution change properties and performing Bayesian inference, unifying various adaptation scenarios.

Findings

Effective on synthetic data

Improves adaptation accuracy

Incorporates prior knowledge

Abstract

This paper is concerned with data-driven unsupervised domain adaptation, where it is unknown in advance how the joint distribution changes across domains, i.e., what factors or modules of the data distribution remain invariant or change across domains. To develop an automated way of domain adaptation with multiple source domains, we propose to use a graphical model as a compact way to encode the change property of the joint distribution, which can be learned from data, and then view domain adaptation as a problem of Bayesian inference on the graphical models. Such a graphical model distinguishes between constant and varied modules of the distribution and specifies the properties of the changes across domains, which serves as prior knowledge of the changing modules for the purpose of deriving the posterior of the target variable $Y$ in the target domain. This provides an end-to-end framework of domain adaptation, in which additional knowledge about how the joint distribution changes, if available, can be directly incorporated to improve the graphical representation. We discuss how causality-based domain adaptation can be put under this umbrella. Experimental results on both synthetic and real data demonstrate the efficacy of the proposed framework for domain adaptation. The code is available at https://github.com/mgong2/DA_Infer .