# On the Bias of Directed Information Estimators

## TL;DR

This paper investigates the assumptions behind directed information estimators for Markov processes, providing conditions for their validity, introducing a partial measure to bound bias, and empirically assessing its significance.

## Contribution

It offers a theoretical characterization of when the Markov assumption holds for directed information estimators and proposes a new partial measure to bound bias in non-Markov cases.

## Key findings

- Conditions for the Markov assumption to hold are characterized using Bayesian network concepts.
- The set of parameters where assumptions fail has Lebesgue measure zero.
- Simulations show the bias bound is often small, indicating estimators are reliable in practice.

## Abstract

When estimating the directed information between two jointly stationary Markov processes, it is typically assumed that the recipient of the directed information is itself Markov of the same order as the joint process. While this assumption is often made explicit in the presentation of such estimators, a characterization of when we can expect the assumption to hold is lacking. Using the concept of d-separation from Bayesian networks, we present sufficient conditions for which this assumption holds. We further show that the set of parameters for which the condition is not also necessary has Lebesgue measure zero. Given the strictness of these conditions, we introduce a notion of partial directed information, which can be used to bound the bias of directed information estimates when the directed information recipient is not itself Markov. Lastly we estimate this bound on simulations in a variety of settings to assess the extent to which the bias should be cause for concern.