Statistical mechanics of error exponents for error-correcting codes

TL;DR

This paper introduces a thermodynamic formalism using the cavity method to compute error exponents of error-correcting codes, revealing phase transitions in code performance over different noise levels.

Contribution

It presents a novel thermodynamic approach and cavity method application to derive average and typical error exponents for LDPC codes on BEC and BSC.

Findings

Derivation of error exponents using the cavity method.

Identification of two phase transitions in code performance.

Differences between average and typical error exponents.

Abstract

Error exponents characterize the exponential decay, when increasing message length, of the probability of error of many error-correcting codes. To tackle the long standing problem of computing them exactly, we introduce a general, thermodynamic, formalism that we illustrate with maximum-likelihood decoding of low-density parity-check (LDPC) codes on the binary erasure channel (BEC) and the binary symmetric channel (BSC). In this formalism, we apply the cavity method for large deviations to derive expressions for both the average and typical error exponents, which differ by the procedure used to select the codes from specified ensembles. When decreasing the noise intensity, we find that two phase transitions take place, at two different levels: a glass to ferromagnetic transition in the space of codewords, and a paramagnetic to glass transition in the space of codes.