Concentration Properties of Random Codes

TL;DR

This paper investigates the concentration of error exponents in random coding for discrete memoryless channels, revealing convergence behaviors and Gaussian-like distributions at low rates, with implications for understanding code performance.

Contribution

It provides new insights into the probabilistic convergence of error exponents in random codes, including at asymptotically low rates and for general ensembles.

Findings

Error exponent converges in probability to its expectation for high rates.

At low rates, the error exponent converges in distribution to a Gaussian-like distribution.

Results extend to generic ensembles and channels, enhancing understanding of code performance.

Abstract

This paper studies the concentration properties of random codes. Specifically, we show that, for discrete memoryless channels, the error exponent of a randomly generated code with pairwise-independent codewords converges in probability to its expectation -- the typical error exponent. For high rates, the result is a consequence of the fact that the random-coding error exponent and the sphere-packing error exponent coincide. For low rates, instead, the convergence is based on the fact that the union bound accurately characterizes the probability of error. The paper also zooms into the behavior at asymptotically low rates and shows that the error exponent converges in distribution to a Gaussian-like distribution. Finally, we present several results on the convergence of the error probability and error exponent for generic ensembles and channels.