Entropy estimates of small data sets

TL;DR

This paper introduces a new balanced entropy estimator optimized for small data sets, reducing bias and errors, especially useful in biological and digital data analysis.

Contribution

A novel entropy estimator designed to improve accuracy for small data sets, balancing bias and variance better than existing methods.

Findings

Outperforms existing estimators on small data sets

Effective when output probabilities are not near zero

Applicable to biological and digital data series

Abstract

Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon, R\'enyi and Tsallis) specially devised to provide a compromise between low bias and small statistical errors, for short data series. This new estimator out-performs other currently available ones when the data sets are small and the probabilities of the possible outputs of the random variable are not close to zero. Otherwise, other well-known estimators remain a better choice. The potential range of applicability of this estimator is quite broad specially for biological and digital data series.