A General Representation-Based Approach to Multi-Source Domain Adaptation

TL;DR

This paper introduces a general framework for multi-source domain adaptation that learns compact latent representations, addressing distribution shifts without restrictive assumptions, and provides theoretical guarantees for transferability.

Contribution

It proposes a novel, theoretically grounded approach that partitions label-related representations to improve transferability across diverse domain shifts.

Findings

Theoretical analysis shows partitioning representations enhances domain adaptation.

The approach can handle various types of distribution shifts.

Identifiability of the learned representations is established.

Abstract

A central problem in unsupervised domain adaptation is determining what to transfer from labeled source domains to an unlabeled target domain. To handle high-dimensional observations (e.g., images), a line of approaches use deep learning to learn latent representations of the observations, which facilitate knowledge transfer in the latent space. However, existing approaches often rely on restrictive assumptions to establish identifiability of the joint distribution in the target domain, such as independent latent variables or invariant label distributions, limiting their real-world applicability. In this work, we propose a general domain adaptation framework that learns compact latent representations to capture distribution shifts relative to the prediction task and address the fundamental question of what representations should be learned and transferred. Notably, we first demonstrate that learning representations based on all the predictive information, i.e., the label's Markov blanket in terms of the learned representations, is often underspecified in general settings. Instead, we show that, interestingly, general domain adaptation can be achieved by partitioning the representations of Markov blanket into those of the label's parents, children, and spouses. Moreover, its identifiability guarantee can be established. Building on these theoretical insights, we develop a practical, nonparametric approach for domain adaptation in a general setting, which can handle different types of distribution shifts.