The random energy model in a magnetic field and joint source-channel coding

TL;DR

This paper reveals a deep connection between the magnetic properties of Derrida's random energy model and joint source-channel coding, linking phase behavior and decoding errors to physical models.

Contribution

It establishes a novel analogy between spin glass physics and information theory, relating REM phases to decoding error patterns and code performance.

Findings

Decoding errors exhibit magnetization-like properties similar to REM phases.

Performance of joint source-channel codes correlates with REM free energy in different phases.

External source distribution acts like a magnetic field in the REM analogy.

Abstract

We demonstrate that there is an intimate relationship between the magnetic properties of Derrida's random energy model (REM) of spin glasses and the problem of joint source--channel coding in Information Theory. In particular, typical patterns of erroneously decoded messages in the coding problem have ``magnetization'' properties that are analogous to those of the REM in certain phases, where the non--uniformity of the distribution of the source in the coding problem, plays the role of an external magnetic field applied to the REM. We also relate the ensemble performance (random coding exponents) of joint source--channel codes to the free energy of the REM in its different phases.