Generalizations of Fano's Inequality for Conditional Information Measures via Majorization Theory

TL;DR

This paper extends Fano's inequality to a wide class of information measures using majorization theory, providing new insights into their asymptotic behavior and linking to the asymptotic equipartition property.

Contribution

It introduces a generalized Fano's inequality applicable to Shannon and Rényi measures through majorization theory, and explores their asymptotic properties.

Findings

Generalized Fano's inequality for multiple information measures

Asymptotic analysis of equivocations with vanishing error probabilities

New characterization of the asymptotic equipartition property

Abstract

Fano's inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano's inequality is generalized to a broad class of information measures, which contains those of Shannon and R\'{e}nyi. When specialized to these measures, it recovers and generalizes the classical inequalities. Key to the derivation is the construction of an appropriate conditional distribution inducing a desired marginal distribution on a countably infinite alphabet. The construction is based on the infinite-dimensional version of Birkhoff's theorem proven by R\'{e}v\'{e}sz [Acta Math. Hungar. 1962, 3, 188{\textendash}198], and the constraint of maintaining a desired marginal distribution is similar to coupling in probability theory. Using our Fano-type inequalities for Shannon's and R\'{e}nyi's information measures, we also investigate the asymptotic behavior of the sequence of Shannon's and R\'{e}nyi's equivocations when the error probabilities vanish. This asymptotic behavior provides a novel characterization of the asymptotic equipartition property (AEP) via Fano's inequality.