Capacity Definitions for General Channels with Receiver Side Information

TL;DR

This paper explores three different capacity definitions for general channels with receiver side information, including Shannon capacity, capacity versus outage, and expected capacity, providing theoretical capacity theorems and discussing their implications.

Contribution

It introduces and analyzes new capacity definitions for non-stationary, non-ergodic channels with receiver side information, extending classical capacity concepts.

Findings

Expected capacity can exceed Shannon capacity for general channels.

Capacity versus outage involves two decoders for different channel states.

Numerical examples illustrate the differences and connections among the capacity definitions.

Abstract

We consider three capacity definitions for general channels with channel side information at the receiver, where the channel is modeled as a sequence of finite dimensional conditional distributions not necessarily stationary, ergodic, or information stable. The {\em Shannon capacity} is the highest rate asymptotically achievable with arbitrarily small error probability. The {\em capacity versus outage} is the highest rate asymptotically achievable with a given probability of decoder-recognized outage. The {\em expected capacity} is the highest average rate asymptotically achievable with a single encoder and multiple decoders, where the channel side information determines the decoder in use. As a special case of channel codes for expected rate, the code for capacity versus outage has two decoders: one operates in the non-outage states and decodes all transmitted information, and the other operates in the outage states and decodes nothing. Expected capacity equals Shannon capacity for channels governed by a stationary ergodic random process but is typically greater for general channels. These alternative capacity definitions essentially relax the constraint that all transmitted information must be decoded at the receiver. We derive capacity theorems for these capacity definitions through information density. Numerical examples are provided to demonstrate their connections and differences. We also discuss the implication of these alternative capacity definitions for end-to-end distortion, source-channel coding and separation.