Confidence Intervals for the Mutual Information

TL;DR

This paper develops distribution-free confidence intervals for mutual information between two variables based on finite samples, combining bounds on distribution differences and deviations without requiring assumptions on the underlying distribution or sample size.

Contribution

It introduces a novel method for constructing confidence intervals for mutual information that do not depend on distributional assumptions or sample size constraints.

Findings

Provides distribution-free confidence intervals for mutual information.

Applicable to finite alphabet variables with arbitrary distributions.

Does not require assumptions on sample size or distribution.

Abstract

By combining a bound on the absolute value of the difference of mutual information between two joint probablity distributions with a fixed variational distance, and a bound on the probability of a maximal deviation in variational distance between a true joint probability distribution and an empirical joint probability distribution, confidence intervals for the mutual information of two random variables with finite alphabets are established. Different from previous results, these intervals do not need any assumptions on the distribution and the sample size.