Relations Between Conditional Shannon Entropy and Expectation of $\ell_{\alpha}$-Norm

TL;DR

This paper explores the precise mathematical relationships and bounds between conditional Shannon entropy and the expectation of the _{\u03b1}-norm, with applications to information measures and channel capacity analysis.

Contribution

It establishes tight bounds between conditional Shannon entropy and _{\u03b1}-norm expectation, extending to various information measures and channel capacity bounds.

Findings

Derived tight bounds between conditional Shannon entropy and _{\u03b1}-norm expectation.

Extended results to conditional Re9nyi entropy and R-norm information.

Applied bounds to analyze Gallager's E0 function under uniform input.

Abstract

The paper examines relationships between the conditional Shannon entropy and the expectation of $\ell_{\alpha}$-norm for joint probability distributions. More precisely, we investigate the tight bounds of the expectation of $\ell_{\alpha}$-norm with a fixed conditional Shannon entropy, and vice versa. As applications of the results, we derive the tight bounds between the conditional Shannon entropy and several information measures which are determined by the expectation of $\ell_{\alpha}$-norm, e.g., the conditional R\'{e}nyi entropy and the conditional $R$-norm information. Moreover, we apply these results to discrete memoryless channels under a uniform input distribution. Then, we show the tight bounds of Gallager's $E_{0}$ functions with a fixed mutual information under a uniform input distribution.