Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains

TL;DR

This paper introduces SYNC, a novel method that learns time-aware causal representations to improve model generalization in evolving domains with distribution shifts over time.

Contribution

It proposes a time-aware structural causal model and a sequential VAE framework to capture evolving causal factors, enhancing temporal generalization.

Findings

SYNC outperforms existing methods on synthetic datasets.

SYNC achieves superior temporal generalization on real-world data.

Theoretical analysis confirms optimal causal predictor derivation.

Abstract

Endowing deep models with the ability to generalize in dynamic scenarios is of vital significance for real-world deployment, given the continuous and complex changes in data distribution. Recently, evolving domain generalization (EDG) has emerged to address distribution shifts over time, aiming to capture evolving patterns for improved model generalization. However, existing EDG methods may suffer from spurious correlations by modeling only the dependence between data and targets across domains, creating a shortcut between task-irrelevant factors and the target, which hinders generalization. To this end, we design a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts, and propose \textbf{S}tatic-D\textbf{YN}amic \textbf{C}ausal Representation Learning (\textbf{SYNC}), an approach that effectively learns time-aware causal representations. Specifically, it integrates specially designed information-theoretic objectives into a sequential VAE framework which captures evolving patterns, and produces the desired representations by preserving intra-class compactness of causal factors both across and within domains. Moreover, we theoretically show that our method can yield the optimal causal predictor for each time domain. Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.