Bias Analysis in Entropy Estimation

TL;DR

This paper introduces new finite sample correction formulas for entropy estimation, analytically computes their bias, and analyzes the trade-off between bias reduction and increased statistical error.

Contribution

It presents novel entropy estimators with analytically derived bias corrections that unify and extend existing correction formulas.

Findings

New entropy estimates with analytically computed bias.

Correction formulas encompass recent literature methods.

Trade-off analysis between bias reduction and statistical error.

Abstract

We consider the problem of finite sample corrections for entropy estimation. New estimates of the Shannon entropy are proposed and their systematic error (the bias) is computed analytically. We find that our results cover correction formulas of current entropy estimates recently discussed in literature. The trade-off between bias reduction and the increase of the corresponding statistical error is analyzed.