An Extended Fano's Inequality for the Finite Blocklength Coding

TL;DR

This paper introduces an extended Fano's inequality that provides tighter bounds for finite blocklength coding, improving the analysis of error probabilities and mutual information in coding theory.

Contribution

The paper proposes a new, tighter version of Fano's inequality tailored for finite blocklength regimes, with applications to symmetric channels.

Findings

The extended Fano's inequality is tighter than the original for finite blocklengths.

Derived bounds on mutual information and codebook size are more accurate.

The inequality is tight for q-ary symmetric channels.

Abstract

Fano's inequality reveals the relation between the conditional entropy and the probability of error . It has been the key tool in proving the converse of coding theorems in the past sixty years. In this paper, an extended Fano's inequality is proposed, which is tighter and more applicable for codings in the finite blocklength regime. Lower bounds on the mutual information and an upper bound on the codebook size are also given, which are shown to be tighter than the original Fano's inequality. Especially, the extended Fano's inequality is tight for some symmetric channels such as the $q$-ary symmetric channels (QSC).