Target Conditioned Representation Independence (TCRI); From Domain-Invariant to Domain-General Representations

TL;DR

This paper introduces TCRI, a novel domain generalization method that enforces conditional independence constraints to learn complete invariant mechanisms, improving worst-case domain performance.

Contribution

TCRI provides a new regularization framework based on conditional independence, addressing limitations of existing methods for better domain generalization.

Findings

Effective on synthetic and real-world data

Competitive average accuracy with improved worst-domain accuracy

Addresses limitations of prior domain generalization methods

Abstract

We propose a Target Conditioned Representation Independence (TCRI) objective for domain generalization. TCRI addresses the limitations of existing domain generalization methods due to incomplete constraints. Specifically, TCRI implements regularizers motivated by conditional independence constraints that are sufficient to strictly learn complete sets of invariant mechanisms, which we show are necessary and sufficient for domain generalization. Empirically, we show that TCRI is effective on both synthetic and real-world data. TCRI is competitive with baselines in average accuracy while outperforming them in worst-domain accuracy, indicating desired cross-domain stability.