Scalable Out-of-distribution Robustness in the Presence of Unobserved Confounders

TL;DR

This paper addresses out-of-distribution generalization challenges caused by unobserved confounders, proposing a simple yet effective predictor under certain assumptions that outperforms existing methods on benchmarks.

Contribution

It introduces a novel approach for OOD robustness with unobserved confounders, requiring only one additional variable for identifiability, simplifying previous complex methods.

Findings

Superior empirical performance on benchmark tasks

Effective handling of unobserved confounders in OOD settings

Simplified predictor requiring only one additional variable

Abstract

We consider the task of out-of-distribution (OOD) generalization, where the distribution shift is due to an unobserved confounder ($Z$) affecting both the covariates ($X$) and the labels ($Y$). This confounding introduces heterogeneity in the predictor, i.e., $P(Y | X) = E_{P(Z | X)}[P(Y | X,Z)]$, making traditional covariate and label shift assumptions unsuitable. OOD generalization differs from traditional domain adaptation in that it does not assume access to the covariate distribution ($X^\text{te}$) of the test samples during training. These conditions create a challenging scenario for OOD robustness: (a) $Z^\text{tr}$ is an unobserved confounder during training, (b) $P^\text{te}(Z) \neq P^\text{tr}(Z)$, (c) $X^\text{te}$ is unavailable during training, and (d) the predictive distribution depends on $P^\text{te}(Z)$. While prior work has developed complex predictors requiring multiple additional variables for identifiability of the latent distribution, we explore a set of identifiability assumptions that yield a surprisingly simple predictor using only a single additional variable. Our approach demonstrates superior empirical performance on several benchmark tasks.