On the effectiveness of a binless entropy estimator for generalised entropic forms

TL;DR

This paper evaluates the Kozachenko-Leonenko entropy estimator's effectiveness across various entropic forms and distributions, highlighting its limitations with non-uniform data and dependent variables.

Contribution

It extends the analysis of the Kozachenko-Leonenko estimator to generalized entropic forms and dependence structures, revealing its domain-specific effectiveness.

Findings

Estimator is effective for uniform distributions across the domain.

Effectiveness diminishes for non-uniform distributions outside the Boltzmann-Gibbs-Shanon form.

Dependence between variables reduces the estimator's accuracy.

Abstract

In this manuscript we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalised to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either linear or non-linear type). We show that when the variables are independently and identically distributed the estimator is only valuable along the whole domain if the data follow the uniform distribution, whereas for other distributions the estimator is only effectual in the limit of the Boltzmann-Gibbs-Shanon entropic form. We also analyse the influence of the dependence (linear and non-linear) between variables on the accuracy of the estimator between variables. As expected in the last case the estimator looses efficiency for the Boltzmann-Gibbs-Shanon entropic form as well.