Non-Asymptotic Error Bounds for Causally Conditioned Directed Information Rates of Gaussian Sequences

TL;DR

This paper derives explicit non-asymptotic error bounds for estimating causally conditioned directed information rates in Gaussian sequences, providing practical formulas and an estimator with quantifiable accuracy.

Contribution

It introduces a new explicit formula for causally conditioned directed information in Gaussian data and analyzes the estimator's error rate, which was previously less understood.

Findings

Estimator achieves an error of order O(N^{-1/2} log(N)) with high probability.

Provides an explicit formula for causally conditioned directed information rate based on optimal prediction.

Extends non-asymptotic error bounds from finite alphabets to Gaussian sequences.

Abstract

Directed information and its causally conditioned variations are often used to measure causal influences between random processes. In practice, these quantities must be measured from data. Non-asymptotic error bounds for these estimates are known for sequences over finite alphabets, but less is known for real-valued data. This paper examines the case in which the data are sequences of Gaussian vectors. We provide an explicit formula for causally conditioned directed information rate based on optimal prediction and define an estimator based on this formula. We show that our estimator gives an error of order $O\left(N^{-1/2}\log(N)\right)$ with high probability, where $N$ is the total sample size.