A Tight Lower Bound on the Mutual Information of a Binary and an Arbitrary Finite Random Variable in Dependence of the Variational Distance

TL;DR

This paper introduces a numerical method to compute a lower bound on the mutual information between a binary and an arbitrary finite random variable, based on the variational distance from a known joint distribution, aiding in mutual information estimation.

Contribution

It provides a novel numerical approach to bound mutual information considering variational distance constraints, enhancing estimation accuracy.

Findings

The method yields tight lower bounds for mutual information.

It enables confidence interval-based mutual information estimation.

Applicable to various joint distributions with known variational distances.

Abstract

In this paper a numerical method is presented, which finds a lower bound for the mutual information between a binary and an arbitrary finite random variable with joint distributions that have a variational distance not greater than a known value to a known joint distribution. This lower bound can be applied to mutual information estimation with confidence intervals.