Estimating the Directed Information and Testing for Causality

TL;DR

This paper develops an asymptotically optimal plug-in estimator for directed information between two processes, linking it to causal influence testing, and provides a likelihood ratio test for causality detection.

Contribution

It introduces the first estimator achieving the optimal convergence rate for directed information and connects estimation with hypothesis testing for causality.

Findings

Estimator is asymptotically Gaussian with $O(1/ ext{sqrt}(n))$ convergence.

Null hypothesis of no causality corresponds to zero directed information.

Likelihood ratio test for causality is derived from the estimator.

Abstract

The problem of estimating the directed information rate between two discrete processes $\{X_n\}$ and $\{Y_n\}$ via the plug-in (or maximum-likelihood) estimator is considered. When the joint process $\{(X_n,Y_n)\}$ is a Markov chain of a given memory length, the plug-in estimator is shown to be asymptotically Gaussian and to converge at the optimal rate $O(1/\sqrt{n})$ under appropriate conditions; this is the first estimator that has been shown to achieve this rate. An important connection is drawn between the problem of estimating the directed information rate and that of performing a hypothesis test for the presence of causal influence between the two processes. Under fairly general conditions, the null hypothesis, which corresponds to the absence of causal influence, is equivalent to the requirement that the directed information rate be equal to zero. In that case a finer result is established, showing that the plug-in converges at the faster rate $O(1/n)$ and that it is asymptotically $\chi^2$-distributed. This is proved by showing that this estimator is equal to (a scalar multiple of) the classical likelihood ratio statistic for the above hypothesis test. Finally it is noted that these results facilitate the design of an actual likelihood ratio test for the presence or absence of causal influence.