Unsupervised domain adaptation with non-stochastic missing data

TL;DR

This paper introduces a generative model for unsupervised domain adaptation that handles missing data in the target domain by performing joint imputation, adaptation, and classification, improving accuracy across various datasets.

Contribution

It proposes a novel joint model for domain adaptation, imputation, and classification that leverages a domain-invariant latent space and indirect supervision from the source.

Findings

The model minimizes an upper bound of target error under various divergence measures.

Joint imputation and adaptation improve classification accuracy in missing data scenarios.

Self-training further enhances the alignment of source and target class distributions.

Abstract

We consider unsupervised domain adaptation (UDA) for classification problems in the presence of missing data in the unlabelled target domain. More precisely, motivated by practical applications, we analyze situations where distribution shift exists between domains and where some components are systematically absent on the target domain without available supervision for imputing the missing target components. We propose a generative approach for imputation. Imputation is performed in a domain-invariant latent space and leverages indirect supervision from a complete source domain. We introduce a single model performing joint adaptation, imputation and classification which, under our assumptions, minimizes an upper bound of its target generalization error and performs well under various representative divergence families (H-divergence, Optimal Transport). Moreover, we compare the target error of our Adaptation-imputation framework and the "ideal" target error of a UDA classifier without missing target components. Our model is further improved with self-training, to bring the learned source and target class posterior distributions closer. We perform experiments on three families of datasets of different modalities: a classical digit classification benchmark, the Amazon product reviews dataset both commonly used in UDA and real-world digital advertising datasets. We show the benefits of jointly performing adaptation, classification and imputation on these datasets.