Extremal Relations Between Shannon Entropy and $\ell_{\alpha}$-Norm

TL;DR

This paper establishes tight bounds between Shannon entropy and the $ ext{l}_eta$-norm for probability vectors, linking these to various information measures and applying results to channels and Gallager's $E_0$ functions.

Contribution

It derives the first tight bounds relating Shannon entropy and $ ext{l}_eta$-norm, and explores their implications for multiple information measures and channel analysis.

Findings

Tight bounds between Shannon entropy and $ ext{l}_eta$-norm are established.

Results connect entropy measures like Rényi and Tsallis to $ ext{l}_eta$-norm bounds.

Applications include bounds on Gallager's $E_0$ functions under fixed mutual information.

Abstract

The paper examines relationships between the Shannon entropy and the $\ell_{\alpha}$-norm for $n$-ary probability vectors, $n \ge 2$. More precisely, we investigate the tight bounds of the $\ell_{\alpha}$-norm with a fixed Shannon entropy, and vice versa. As applications of the results, we derive the tight bounds between the Shannon entropy and several information measures which are determined by the $\ell_{\alpha}$-norm, e.g., R\'{e}nyi entropy, Tsallis entropy, the $R$-norm information, and some diversity indices. Moreover, we apply these results to uniformly focusing channels. Then, we show the tight bounds of Gallager's $E_{0}$ functions with a fixed mutual information under a uniform input distribution.