Statistical mechanical analysis of a hierarchical random code ensemble in signal processing

TL;DR

This paper applies statistical mechanics, specifically the replica method, to analyze a hierarchical random code ensemble's performance in data compression and channel coding, revealing phase transitions and the importance of replica symmetry breaking.

Contribution

It introduces a novel analysis of hierarchical random codes using the generalized random energy model and explores phase transitions via replica symmetry breaking.

Findings

Performance exponents are derived from a generating function.

Transitions in exponents are interpreted as phase transitions.

Replica symmetry breaking is crucial for understanding these transitions.

Abstract

We study a random code ensemble with a hierarchical structure, which is closely related to the generalized random energy model with discrete energy values. Based on this correspondence, we analyze the hierarchical random code ensemble by using the replica method in two situations: lossy data compression and channel coding. For both the situations, the exponents of large deviation analysis characterizing the performance of the ensemble, the distortion rate of lossy data compression and the error exponent of channel coding in Gallager's formalism, are accessible by a generating function of the generalized random energy model. We discuss that the transitions of those exponents observed in the preceding work can be interpreted as phase transitions with respect to the replica number. We also show that the replica symmetry breaking plays an essential role in these transitions.