Strong replica symmetry in high-dimensional optimal Bayesian inference

TL;DR

This paper proves strong replica symmetry in high-dimensional Bayesian inference models by establishing concentration of multioverlaps through novel identities derived from side-observations, simplifying the posterior structure.

Contribution

It introduces a new perturbation method using exponential side-observations to prove replica symmetry in Bayesian inference models.

Findings

Replica symmetry implies simple posterior structure in high dimensions.

The Franz-de Sanctis identities are derived via novel perturbations.

Concentration of multioverlaps supports rigorous derivation of free energy formulas.

Abstract

We consider generic optimal Bayesian inference, namely, models of signal reconstruction where the posterior distribution and all hyperparameters are known. Under a standard assumption on the concentration of the free energy, we show how replica symmetry in the strong sense of concentration of all multioverlaps can be established as a consequence of the Franz-de Sanctis identities; the identities themselves in the current setting are obtained via a novel perturbation coming from exponentially distributed "side-observations" of the signal. Concentration of multioverlaps means that asymptotically the posterior distribution has a particularly simple structure encoded by a random probability measure (or, in the case of binary signal, a non-random probability measure). We believe that such strong control of the model should be key in the study of inference problems with underlying sparse graphical structure (error correcting codes, block models, etc) and, in particular, in the rigorous derivation of replica symmetric formulas for the free energy and mutual information in this context.