On Generalized Sch\"urmann Entropy Estimators

TL;DR

This paper introduces a new class of entropy estimators that are unbiased and have finite variance for severely undersampled discrete distributions, improving accuracy over existing methods.

Contribution

It generalizes a previous estimator by Schuermann, achieving bias-free and finite variance estimators under certain parameters, with extensive numerical validation.

Findings

Estimators are unbiased and have finite variance for specific parameters.

Comparison shows improved performance over existing estimators.

Identifies conflicts with Bayesian estimators for mutual information.

Abstract

We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed by myself in arXiv:physics/0307138. For a special set of parameters they are completely free of bias and have a finite variance, something with is widely believed to be impossible. We present also detailed numerical tests where we compare them with other recent estimators and with exact results, and point out a clash with Bayesian estimators for mutual information.