Shannon Information Theory Without Shedding Tears Over Delta \& Epsilon Proofs or Typical Sequences

TL;DR

This paper revisits fundamental Shannon Information Theory problems using complex integration techniques and Bayesian networks, offering alternative proofs that avoid traditional delta-epsilon and typical sequences methods.

Contribution

It introduces a novel approach to classical SIT proofs employing complex integration and CB nets, providing new perspectives and techniques.

Findings

Proves classical results using steepest descent integration methods.

Demonstrates benefits of Bayesian networks in classical SIT.

Offers alternative, non-traditional proofs for source and channel coding.

Abstract

This paper begins with a discussion of integration over probability types (p-types). After doing that, the paper re-visits 3 mainstay problems of classical (non-quantum) Shannon Information Theory (SIT): source coding without distortion, channel coding, and source coding with distortion. The paper proves well-known, conventional results for each of these 3 problems. However, the proofs given for these results are not conventional. They are based on complex integration techniques (approximations obtained by applying the method of steepest descent to p-type integrals) instead of the usual delta & epsilon and typical sequences arguments. Another unconventional feature of this paper is that we make ample use of classical Bayesian networks (CB nets). This paper showcases some of the benefits of using CB nets to do classical SIT.