CausalFormer: An Interpretable Transformer for Temporal Causal Discovery

TL;DR

CausalFormer is an interpretable transformer model designed for temporal causal discovery, leveraging a causality-aware transformer and a decomposition-based detector to uncover causal relations in time series data with state-of-the-art accuracy.

Contribution

The paper introduces CausalFormer, a novel interpretable transformer architecture that comprehensively utilizes deep learning components for causal discovery in time series.

Findings

Achieves state-of-the-art performance on synthetic, simulated, and real datasets.

Effectively interprets causal relations using regression relevance propagation.

Outperforms existing methods in temporal causal discovery tasks.

Abstract

Temporal causal discovery is a crucial task aimed at uncovering the causal relations within time series data. The latest temporal causal discovery methods usually train deep learning models on prediction tasks to uncover the causality between time series. They capture causal relations by analyzing the parameters of some components of the trained models, e.g., attention weights and convolution weights. However, this is an incomplete mapping process from the model parameters to the causality and fails to investigate the other components, e.g., fully connected layers and activation functions, that are also significant for causal discovery. To facilitate the utilization of the whole deep learning models in temporal causal discovery, we proposed an interpretable transformer-based causal discovery model termed CausalFormer, which consists of the causality-aware transformer and the decomposition-based causality detector. The causality-aware transformer learns the causal representation of time series data using a prediction task with the designed multi-kernel causal convolution which aggregates each input time series along the temporal dimension under the temporal priority constraint. Then, the decomposition-based causality detector interprets the global structure of the trained causality-aware transformer with the proposed regression relevance propagation to identify potential causal relations and finally construct the causal graph. Experiments on synthetic, simulated, and real datasets demonstrate the state-of-the-art performance of CausalFormer on discovering temporal causality. Our code is available at https://github.com/lingbai-kong/CausalFormer.