Distributionally Robust Learning for Multi-source Unsupervised Domain Adaptation

TL;DR

This paper proposes a distributionally robust learning method for multi-source unsupervised domain adaptation, improving generalization to target domains with distribution shifts by aggregating source models and ensuring privacy.

Contribution

It introduces a novel distributionally robust model that aggregates source domain models and includes a bias-correction step, applicable with various machine learning algorithms.

Findings

Effective in simulated data

Improves target domain prediction accuracy

Compatible with multiple ML algorithms

Abstract

Empirical risk minimization often performs poorly when the distribution of the target domain differs from those of source domains. To address such potential distribution shifts, we develop an unsupervised domain adaptation approach that leverages labeled data from multiple source domains and unlabeled data from the target domain. We introduce a distributionally robust model that optimizes an adversarial reward based on the explained variance across a class of target distributions, ensuring generalization to the target domain. We show that the proposed robust model is a weighted average of conditional outcome models from source domains. This formulation allows us to compute the robust model through the aggregation of source models, which can be estimated using various machine learning algorithms of the users' choice, such as random forests, boosting, and neural networks. Additionally, we introduce a bias-correction step to obtain a more accurate aggregation weight, which is effective for various machine learning algorithms. Our framework can be interpreted as a distributionally robust federated learning approach that satisfies privacy constraints while providing insights into the importance of each source for prediction on the target domain. The performance of our method is evaluated on both simulated and real data.