Towards a Framework for Observational Causality From Time Series: When Shannon Meets Turing

TL;DR

This paper introduces a tensor-based framework for causal inference from time series, linking information theory and channel models to distinguish direct and indirect causal relations.

Contribution

It presents a novel tensor formalism for causal analysis, demonstrating that bivariate analysis can differentiate causal types when combined with single-channel data.

Findings

Transfer entropy results from information transmission over channels.

Bivariate analysis suffices to distinguish direct and indirect relations.

A Data Processing Inequality for transfer entropy is established.

Abstract

We propose a novel tensor-based formalism for inferring causal structures from time series. An information theoretical analysis of transfer entropy, shows that transfer entropy results from transmission of information over a set of communication channels. Tensors are the mathematical equivalents of these multi-channel causal channels. A multi-channel causal channel is a generalization of a discrete memoryless channel. Investigation of a system comprising three variables shows that in our formalism, bivariate analysis suffices to differentiate between direct and indirect relations. For this to be true, we have to combine the output of multi-channel causal channels with the output of single-channel causal channels. We can understand this result when we consider the role of noise. Subsequent transmission of information over noisy channels can never result in less noisy transmission overall. This implies that a Data Processing Inequality exists for transfer entropy.