Bayesian Error Based Sequences of Mutual Information Bounds

TL;DR

This paper develops a sequence of tighter bounds on mutual information and Bayesian error by refining existing inequalities, providing improved tools for analyzing Bayesian inference problems.

Contribution

It introduces finite sequences of bounds on mutual information and error probabilities, enhancing the precision of Bayesian error analysis beyond traditional bounds.

Findings

Sequences of bounds tighten existing inequalities.

Application to spatial localization and spectral typing.

Improved understanding of mutual information and error relations.

Abstract

The inverse relation between mutual information (MI) and Bayesian error is sharpened by deriving finite sequences of upper and lower bounds on MI in terms of the minimum probability of error (MPE) and related Bayesian quantities. The well known Fano upper bound and Feder-Merhav lower bound on equivocation are tightened by including a succession of posterior probabilities starting at the largest, which directly controls the MPE, and proceeding to successively lower ones. A number of other interesting results are also derived, including a sequence of upper bounds on the MPE in terms of a previously introduced sequence of generalized posterior distributions. The tightness of the various bounds is illustrated for a simple application of joint spatial localization and spectral typing of a point source.