On the Computation of the Shannon Capacity of a Discrete Channel with Noise

TL;DR

This paper clarifies and corrects the formula for the Shannon capacity of a binary noisy channel, providing a measure-theoretic proof and an explicit expression dependent only on channel parameters.

Contribution

It offers a corrected, clarified formula for binary channel capacity and an alternative proof of the optimal input distribution's feasibility using measure theory.

Findings

Corrected the capacity formula for binary channels

Provided an explicit capacity expression dependent on parameters

Presented a measure-theoretic proof of optimal input distribution

Abstract

Muroga [M52] showed how to express the Shannon channel capacity of a discrete channel with noise [S49] as an explicit function of the transition probabilities. His method accommodates channels with any finite number of input symbols, any finite number of output symbols and any transition probability matrix. Silverman [S55] carried out Muroga's method in the special case of a binary channel (and went on to analyse "cascades" of several such binary channels). This article is a note on the resulting formula for the capacity C(a, c) of a single binary channel. We aim to clarify some of the arguments and correct a small error. In service of this aim, we first formulate several of Shannon's definitions and proofs in terms of discrete measure-theoretic probability theory. We provide an alternate proof to Silverman's, of the feasibility of the optimal input distribution for a binary channel. For convenience, we also express C(a, c) in a single expression explicitly dependent on a and c only, which Silverman stopped short of doing.