Transfer Entropy as a Log-likelihood Ratio

TL;DR

This paper demonstrates that transfer entropy can be estimated using a log-likelihood ratio test, linking it to causality measures like Granger causality, with broad applicability across stochastic models.

Contribution

It establishes that the log-likelihood ratio test statistic is a consistent estimator for transfer entropy in a wide class of models, generalizing Gaussian case results and connecting information transfer with causality.

Findings

The log-likelihood ratio test estimates transfer entropy consistently.

For finite Markov chains, no explicit model is needed.

Asymptotic chi-squared distribution is derived for the estimator.

Abstract

Transfer entropy, an information-theoretic measure of time-directed information transfer between joint processes, has steadily gained popularity in the analysis of complex stochastic dynamics in diverse fields, including the neurosciences, ecology, climatology and econometrics. We show that for a broad class of predictive models, the log-likelihood ratio test statistic for the null hypothesis of zero transfer entropy is a consistent estimator for the transfer entropy itself. For finite Markov chains, furthermore, no explicit model is required. In the general case, an asymptotic chi-squared distribution is established for the transfer entropy estimator. The result generalises the equivalence in the Gaussian case of transfer entropy and Granger causality, a statistical notion of causal influence based on prediction via vector autoregression, and establishes a fundamental connection between directed information transfer and causality in the Wiener-Granger sense.