The Bethe Free Energy Allows to Compute the Conditional Entropy of Graphical Code Instances. A Proof from the Polymer Expansion

TL;DR

This paper demonstrates that the Bethe free energy accurately approximates the conditional entropy of certain graphical error-correcting codes in large noise regimes, using advanced statistical mechanics techniques.

Contribution

It introduces a novel application of polymer expansion techniques to relate Bethe free energy to true free energy in graphical models, especially for parity-check codes.

Findings

Bethe free energy is asymptotically exact for LDPC codes in high noise regimes.

Develops new methods combining polymer expansion with graphical code analysis.

Shows the difference between true and Bethe free energies vanishes with high probability.

Abstract

The main objective of this paper is to explore the precise relationship between the Bethe free energy (or entropy) and the Shannon conditional entropy of graphical error correcting codes. The main result shows that the Bethe free energy associated with a low-density parity-check code used over a binary symmetric channel in a large noise regime is, with high probability, asymptotically exact as the block length grows. To arrive at this result we develop new techniques for rather general graphical models based on the loop sum as a starting point and the polymer expansion from statistical mechanics. The true free energy is computed as a series expansion containing the Bethe free energy as its zero-th order term plus a series of corrections. It is easily seen that convergence criteria for such expansions are satisfied for general high-temperature models. We apply these general results to ensembles of low-density generator-matrix and parity-check codes. While the application to generator-matrix codes follows standard "high temperature" methods, the case of parity-check codes requires non-trivial new ideas because the hard constraints correspond to a zero-temperature regime. Nevertheless one can combine the polymer expansion with expander and counting arguments to show that the difference between the true and Bethe free energies vanishes with high probability in the large block