A Statistical-Physics Refinement of Soft Covering

TL;DR

This paper applies statistical physics to analyze the output distribution of random codes, revealing phase transitions and competition between typical and atypical codeword populations, with applications to information theory tasks.

Contribution

It derives a novel two-branch free energy formula capturing the competition between typical and atypical codewords in channel output distributions.

Findings

Identifies phase boundaries and a phase diagram for the code ensemble.

Derives explicit formulas for the phase boundary at $eta \,\geq\, 1$.

Illustrates results with a numerical example of a Z-channel.

Abstract

We study the channel output distribution induced by a rate-$R$ random code via statistical physics. The partition function is $Z_n(\beta|\mathcal{C}) = \sum_{y^n}[P_{Y^n|\mathcal{C}}(y^n)]^\beta$, where $\mathcal{C}$ is the code and $\beta > 0$ is inverse temperature. Our focus is on the free energy which is the normalized logarithm of this quantity, which encodes the full R\'{e}nyi spectrum of the output distribution. The single-letter formula derived for the annealed free energy decomposes into two branches which reflect a ``competition'' between two populations of codewords. One is the \emph{bulk branch}, $\psi_{\mbox{\tiny b}}(\beta,R)$, which is driven by typical codewords and the other one is the \emph{sparse branch} $\psi_{\mbox{\tiny s}}(\beta,R)$, which is driven by a-typical codewords, where the qualifiers `typical' and `atypical' are in a sense to become apparent later. We analyze the phase structure of each branch separately and characterize their competition. Both branches are derived for all $\beta > 0$. The phase boundary $R^\star(\beta)$, where the two branches are equal, is analyzed for $\beta \geq 1$, where it has an explicit closed-form expression. The phase diagram in the first quadrant of the $(\beta, R)$ plane has four regions separated by three boundaries: $R = I^{\mbox{\tiny b}}(\beta)$ (bulk branch transition), $R = R^\star(\beta)$ (bulk--sparse competition boundary), and $R = I^{\mbox{\tiny s}}(\beta)$ (sparse branch transition), all meeting at the point $(\beta, R) = (1, I(X;Y))$, where $I(X;Y)$ is the mutual information induced by the input type and the channel. Applications to guesswork, channel resolvability, and hypothesis testing are discussed, and all results are illustrated with a numerical example of a Z-channel.