Towards Identifiability of Hierarchical Temporal Causal Representation Learning

TL;DR

This paper introduces a new framework called CHiLD for identifying hierarchical temporal causal structures in time series data, overcoming limitations of existing methods by leveraging three conditionally independent observations.

Contribution

We propose a novel identifiability framework for hierarchical latent dynamics that uses three observations and develop a variational inference model for practical implementation.

Findings

Successfully identifies hierarchical latent structures in synthetic data.

Effectively models real-world time series with hierarchical dynamics.

Outperforms existing methods in capturing multi-level temporal dependencies.

Abstract

Modeling hierarchical latent dynamics behind time series data is critical for capturing temporal dependencies across multiple levels of abstraction in real-world tasks. However, existing temporal causal representation learning methods fail to capture such dynamics, as they fail to recover the joint distribution of hierarchical latent variables from \textit{single-timestep observed variables}. Interestingly, we find that the joint distribution of hierarchical latent variables can be uniquely determined using three conditionally independent observations. Building on this insight, we propose a Causally Hierarchical Latent Dynamic (CHiLD) identification framework. Our approach first employs temporal contextual observed variables to identify the joint distribution of multi-layer latent variables. Sequentially, we exploit the natural sparsity of the hierarchical structure among latent variables to identify latent variables within each layer. Guided by the theoretical results, we develop a time series generative model grounded in variational inference. This model incorporates a contextual encoder to reconstruct multi-layer latent variables and normalize flow-based hierarchical prior networks to impose the independent noise condition of hierarchical latent dynamics. Empirical evaluations on both synthetic and real-world datasets validate our theoretical claims and demonstrate the effectiveness of CHiLD in modeling hierarchical latent dynamics.