Transfer entropy for finite data

TL;DR

This paper introduces a new transfer entropy measure tailored for finite, sparse data that reduces bias and enables nonparametric significance testing, improving analysis of small or high-cardinality time series.

Contribution

It proposes a transfer entropy estimator that addresses bias and significance assessment issues in finite data scenarios, without relying on simulations.

Findings

Reduces positive bias in transfer entropy estimates for sparse data.

Allows nonparametric significance testing without simulation.

Effective for small or high-cardinality time series.

Abstract

Transfer entropy is a widely used measure for quantifying directed information flows in complex systems. While the challenges of estimating transfer entropy for continuous data are well known, it has two major shortcomings for data of finite cardinality: it exhibits a substantial positive bias for sparse bin counts, and it has no clear means to assess statistical significance. By computing information content in finite data streams without explicitly considering symbols as instances of random variables, we derive a transfer entropy measure which is asymptotically equivalent to the standard plug-in estimator but remedies these issues for time series of small size and/or high cardinality, permitting a fully nonparametric assessment of statistical significance without simulation.