Anti-causal domain generalization: Leveraging unlabeled data

TL;DR

This paper introduces methods for domain generalization in anti-causal settings, leveraging unlabeled data to regularize models against environment-induced covariate shifts, with proven optimality and empirical validation.

Contribution

It proposes two novel regularization techniques that utilize unlabeled data to improve robustness in anti-causal domain generalization, with theoretical guarantees.

Findings

Methods outperform baselines on physical system data.

Techniques effectively leverage unlabeled data for robustness.

Proven worst-case optimality under certain environment classes.

Abstract

The problem of domain generalization concerns learning predictive models that are robust to distribution shifts when deployed in new, previously unseen environments. Existing methods typically require labeled data from multiple training environments, limiting their applicability when labeled data are scarce. In this work, we study domain generalization in an anti-causal setting, where the outcome causes the observed covariates. Under this structure, environment perturbations that affect the covariates do not propagate to the outcome, which motivates regularizing the model's sensitivity to these perturbations. Crucially, estimating these perturbation directions does not require labels, enabling us to leverage unlabeled data from multiple environments. We propose two methods that penalize the model's sensitivity to variations in the mean and covariance of the covariates across environments, respectively, and prove that these methods have worst-case optimality guarantees under certain classes of environments. Finally, we demonstrate the empirical performance of our approach on a controlled physical system and a physiological signal dataset.