Capacity and Random-Coding Exponents for Channel Coding with Side Information

TL;DR

This paper derives capacity formulas and random-coding exponents for a generalized Gel'fand-Pinsker channel model with side information, adversarial channel selection, and cost constraints, extending to channels with memory and applications like watermarking.

Contribution

It introduces new capacity and exponent results for channels with side information, including channels with memory, using a stacked binning scheme and a generalized decoder.

Findings

Random-coding exponents are larger for channels with arbitrary memory.

The proposed coding scheme achieves asymptotic error bounds.

Applications include watermarking and data hiding.

Abstract

Capacity formulas and random-coding exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. These exponents yield asymptotic upper bounds on the achievable log probability of error. In our model, information is to be reliably transmitted through a noisy channel with finite input and output alphabets and random state sequence, and the channel is selected by a hypothetical adversary. Partial information about the state sequence is available to the encoder, adversary, and decoder. The design of the transmitter is subject to a cost constraint. Two families of channels are considered: 1) compound discrete memoryless channels (CDMC), and 2) channels with arbitrary memory, subject to an additive cost constraint, or more generally to a hard constraint on the conditional type of the channel output given the input. Both problems are closely connected. The random-coding exponent is achieved using a stacked binning scheme and a maximum penalized mutual information decoder, which may be thought of as an empirical generalized Maximum a Posteriori decoder. For channels with arbitrary memory, the random-coding exponents are larger than their CDMC counterparts. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning.