Distributionally Robust Classification for Multi-source Unsupervised Domain Adaptation

TL;DR

This paper introduces a distributionally robust learning framework for unsupervised domain adaptation that models uncertainty in data distributions, improving model generalization especially with scarce target data.

Contribution

It proposes a novel distributionally robust approach applicable to multi-source and single-source UDA, with an efficient algorithm that enhances existing methods.

Findings

Outperforms strong baselines in various distribution shift scenarios.

Shows significant improvements when target data are scarce.

Robust against spurious correlations and limited unlabeled data.

Abstract

Unsupervised domain adaptation (UDA) is a statistical learning problem when the distribution of training (source) data is different from that of test (target) data. In this setting, one has access to labeled data only from the source domain and unlabeled data from the target domain. The central objective is to leverage the source data and the unlabeled target data to build models that generalize to the target domain. Despite its potential, existing UDA approaches often struggle in practice, particularly in scenarios where the target domain offers only limited unlabeled data or spurious correlations dominate the source domain. To address these challenges, we propose a novel distributionally robust learning framework that models uncertainty in both the covariate distribution and the conditional label distribution. Our approach is motivated by the multi-source domain adaptation setting but is also directly applicable to the single-source scenario, making it versatile in practice. We develop an efficient learning algorithm that can be seamlessly integrated with existing UDA methods. Extensive experiments under various distribution shift scenarios show that our method consistently outperforms strong baselines, especially when target data are extremely scarce.