Causal Temporal Representation Learning with Nonstationary Sparse Transition

TL;DR

This paper introduces CtrlNS, a novel framework for causal temporal representation learning that leverages sparse transition assumptions to identify distribution shifts without prior domain variable knowledge.

Contribution

It provides theoretical identifiability results under sparse transition assumptions and develops a new method that outperforms existing baselines on synthetic and real data.

Findings

Significant improvement over baselines in experiments

Theoretical guarantees for identifying distribution shifts

Effective in real-world nonstationary temporal sequences

Abstract

Causal Temporal Representation Learning (Ctrl) methods aim to identify the temporal causal dynamics of complex nonstationary temporal sequences. Despite the success of existing Ctrl methods, they require either directly observing the domain variables or assuming a Markov prior on them. Such requirements limit the application of these methods in real-world scenarios when we do not have such prior knowledge of the domain variables. To address this problem, this work adopts a sparse transition assumption, aligned with intuitive human understanding, and presents identifiability results from a theoretical perspective. In particular, we explore under what conditions on the significance of the variability of the transitions we can build a model to identify the distribution shifts. Based on the theoretical result, we introduce a novel framework, Causal Temporal Representation Learning with Nonstationary Sparse Transition (CtrlNS), designed to leverage the constraints on transition sparsity and conditional independence to reliably identify both distribution shifts and latent factors. Our experimental evaluations on synthetic and real-world datasets demonstrate significant improvements over existing baselines, highlighting the effectiveness of our approach.