Invariant and Transportable Representations for Anti-Causal Domain Shifts

TL;DR

This paper introduces a method for learning invariant and transportable representations in domain shift scenarios, especially addressing anti-causal problems where traditional causal invariance does not apply, enabling better generalization and adaptation.

Contribution

It proposes a novel representation learning approach that disentangles invariant and non-stable features under anti-causal domain shifts, improving transferability.

Findings

Effective in synthetic and real-world datasets

Enables fast adaptation to new domains

Outperforms baseline methods in domain generalization

Abstract

Real-world classification problems must contend with domain shift, the (potential) mismatch between the domain where a model is deployed and the domain(s) where the training data was gathered. Methods to handle such problems must specify what structure is common between the domains and what varies. A natural assumption is that causal (structural) relationships are invariant in all domains. Then, it is tempting to learn a predictor for label $Y$ that depends only on its causal parents. However, many real-world problems are "anti-causal" in the sense that $Y$ is a cause of the covariates $X$ -- in this case, $Y$ has no causal parents and the naive causal invariance is useless. In this paper, we study representation learning under a particular notion of domain shift that both respects causal invariance and that naturally handles the "anti-causal" structure. We show how to leverage the shared causal structure of the domains to learn a representation that both admits an invariant predictor and that also allows fast adaptation in new domains. The key is to translate causal assumptions into learning principles that disentangle "invariant" and "non-stable" features. Experiments on both synthetic and real-world data demonstrate the effectiveness of the proposed learning algorithm. Code is available at https://github.com/ybjiaang/ACTIR.