A note on the unbiased estimation of mutual information

TL;DR

This paper discusses the bias in mutual information estimators and shows how to explicitly calculate the bias for the Kozachenko-Leonenko estimator in metric spaces.

Contribution

It provides a method to explicitly compute the bias of the Kozachenko-Leonenko mutual information estimator in metric spaces.

Findings

Bias of the Kozachenko-Leonenko estimator can be explicitly calculated.

Explicit bias formulas improve the understanding of estimator accuracy.

Potential for bias correction in mutual information estimation.

Abstract

Estimators for mutual information are typically biased. However, in the case of the Kozachenko-Leonenko estimator for metric spaces, a type of nearest neighbour estimator, it is possible to calculate the bias explicitly.