The Generalized Random Energy Model and its Application to the Statistical Physics of Ensembles of Hierarchical Codes

TL;DR

This paper explores the generalized random energy model (GREM) from statistical physics and applies its hierarchical structure insights to the analysis and design of hierarchical code ensembles, revealing phase transition behaviors.

Contribution

It extends the analogy between GREM and code ensembles, analyzing phase transitions and hierarchical effects for improved code design insights.

Findings

Hierarchical code ensembles exhibit phase transition behaviors similar to GREM.

Different design parameters lead to substantially different behaviors in code performance.

Insights from GREM can inform code design and optimization strategies.

Abstract

In an earlier work, the statistical physics associated with finite--temperature decoding of code ensembles, along with the relation to their random coding error exponents, were explored in a framework that is analogous to Derrida's random energy model (REM) of spin glasses, according to which the energy levels of the various spin configurations are independent random variables. The generalized REM (GREM) extends the REM in that it introduces correlations between energy levels in an hierarchical structure. In this paper, we explore some analogies between the behavior of the GREM and that of code ensembles which have parallel hierarchical structures. In particular, in analogy to the fact that the GREM may have different types of phase transition effects, depending on the parameters of the model, then the above--mentioned hierarchical code ensembles behave substantially differently in the various domains of the design parameters of these codes. We make an attempt to explore the insights that can be imported from the statistical mechanics of the GREM and be harnessed to serve for code design considerations and guidelines.