Large Deviations Behavior of the Logarithmic Error Probability of Random Codes

TL;DR

This paper analyzes the probability of deviations in the error exponent of random codes, showing that significant deviations are exponentially or double-exponentially unlikely, with bounds derived using new large deviations results.

Contribution

It introduces new large deviations bounds for the error exponents of random codes, especially for deviations above the typical error exponent, with tight bounds in some cases.

Findings

Probability of lower-than-TRC error exponent is exponentially small.

Probability of higher-than-TRC error exponent is double-exponentially small.

Provides bounds for these deviation probabilities, with some bounds coinciding.

Abstract

This work studies the deviations of the error exponent of the constant composition code ensemble around its expectation, known as the error exponent of the typical random code (TRC). In particular, it is shown that the probability of randomly drawing a codebook whose error exponent is smaller than the TRC exponent is exponentially small; upper and lower bounds for this exponent are given, which coincide in some cases. In addition, the probability of randomly drawing a codebook whose error exponent is larger than the TRC exponent is shown to be double-exponentially small; upper and lower bounds to the double-exponential exponent are given. The results suggest that codebooks whose error exponent is larger than the error exponent of the TRC are extremely rare. The key ingredient in the proofs is a new large deviations result of type class enumerators with dependent variables.