On the Practical Estimation and Interpretation of R\'enyi Transfer Entropy

TL;DR

This paper evaluates a k-nearest neighbor estimator for Re9nyi transfer entropy, providing practical guidelines for its estimation and interpretation in complex, high-dimensional, and heterogeneous datasets, highlighting its sensitivity to the parameter e9.

Contribution

It systematically studies the estimator's performance, introduces reliability conditions, and offers calibration guidelines to improve RTE's practical application in complex systems.

Findings

Effective RTE estimates require meeting reliability conditions.

RTE's sensitivity to e9 highlights its potential for extreme event analysis.

Calibration of estimator parameters improves directional information flow detection.

Abstract

R\'enyi transfer entropy (RTE) is a generalization of classical transfer entropy that replaces Shannon's entropy with R\'enyi's information measure. This, in turn, introduces a new tunable parameter $\alpha$, which accounts for sensitivity to low- or high-probability events. Although RTE shows strong potential for analyzing causal relations in complex, non-Gaussian systems, its practical use is limited, primarily due to challenges related to its accurate estimation and interpretation. These difficulties are especially pronounced when working with finite, high-dimensional, or heterogeneous datasets. In this paper, we systematically study the performance of a k-nearest neighbor estimator for both R\'enyi entropy (RE) and RTE using various synthetic data sets with clear cause-and-effect relationships inherent to their construction. We test the estimator across a broad range of parameters, including sample size, dimensionality, memory length, and R\'enyi order $\alpha$. In particular, we apply the estimator to a set of simulated processes with increasing structural complexity, ranging from linear dynamics to nonlinear systems with multi-source couplings. To address interpretational challenges arising from potentially negative RE and RTE values, we introduce three reliability conditions and formulate practical guidelines for tuning the estimator parameters. We show that when the reliability conditions are met and the parameters are calibrated accordingly, the resulting effective RTE estimates accurately capture directional information flow across a broad range of scenarios. Results obtained show that the explanatory power of RTE depends sensitively on the choice of the R\'{e}nyi parameter $\alpha$. This highlights the usefulness of the RTE framework for identifying the drivers of extreme behavior in complex systems.