An Elementary Proof of a Classical Information-Theoretic Formula

TL;DR

This paper provides a straightforward, elementary proof of Shannon's classical formula for the mutual information rate of a Gaussian channel, avoiding complex mathematics and relying on basic calculus and matrix theory.

Contribution

It introduces a rigorous, self-contained proof of Shannon's formula using a novel sampling theorem, simplifying previous complex approaches.

Findings

Elementary proof of Shannon's formula established

Relies only on basic calculus and matrix theory

Proof is rigorous and self-contained

Abstract

A renowned information-theoretic formula by Shannon expresses the mutual information rate of a white Gaussian channel with a stationary Gaussian input as an integral of a simple function of the power spectral density of the channel input. We give in this paper a rigorous yet elementary proof of this classical formula. As opposed to all the conventional approaches, which either rely on heavy mathematical machineries or have to resort to some "external" results, our proof, which hinges on a recently proven sampling theorem, is elementary and self-contained, only using some well-known facts from basic calculus and matrix theory.