Estimating Functions of Distributions from A Finite Set of Samples, Part 2: Bayes Estimators for Mutual Information, Chi-Squared, Covariance and other Statistics

TL;DR

This paper introduces finite sample estimators for entropy, mutual information, covariance, and chi-squared functions of discrete distributions, enabling more accurate statistical analysis from limited data samples.

Contribution

It provides novel Bayes estimators for mutual information, covariance, and chi-squared statistics based on finite samples, extending previous work on entropy estimation.

Findings

Effective estimators for mutual information and covariance from finite samples.

Improved accuracy over traditional estimators in limited data scenarios.

Applicability to discrete joint distributions in various statistical analyses.

Abstract

We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint distribution, we present finite sample estimators for the mutual information, covariance, and chi-squared functions of that probability distribution.