Error exponents of typical random codes

TL;DR

This paper introduces the concept of the error exponent of the typical random code, providing exact calculations for discrete memoryless channels and generalized decoders, and explores its relation to traditional error exponents.

Contribution

It defines and analyzes the error exponent of the typical random code for general DMCs and decoders, extending understanding beyond traditional random coding exponents.

Findings

Exact error exponents for typical random codes are derived.

The fixed composition code ensemble is shown to be optimal among permutation-invariant distributions.

The analysis applies to list decoding and erasure/list options.

Abstract

We define the error exponent of the typical random code as the long-block limit of the negative normalized expectation of the logarithm of the error probability of the random code, as opposed to the traditional random coding error exponent, which is the limit of the negative normalized logarithm of the expectation of the error probability. For the ensemble of uniformly randomly drawn fixed composition codes, we provide exact error exponents of typical random codes for a general discrete memoryless channel (DMC) and a wide class of (stochastic) decoders, collectively referred to as the generalized likelihood decoder (GLD). This ensemble of fixed composition codes is shown to be no worse than any other ensemble of independent codewords that are drawn under a permutation--invariant distribution (e.g., i.i.d. codewords). We also present relationships between the error exponent of the typical random code and the ordinary random coding error exponent, as well as the expurgated exponent for the GLD. Finally, we demonstrate that our analysis technique is applicable also to more general communication scenarios, such as list decoding (for fixed-size lists) as well as decoding with an erasure/list option in Forney's sense.