Differentiability and overlap concentration in optimal Bayesian inference

TL;DR

This paper investigates the properties of Bayesian inference models for finite-rank tensor products, demonstrating that at differentiable points of the free energy, the overlap concentrates and the MMSE converges, indicating replica symmetry.

Contribution

It establishes the connection between differentiability of free energy and overlap concentration in Bayesian tensor models, extending understanding of replica symmetry at various signal-to-noise ratios.

Findings

Overlap concentrates at the free energy gradient.

MMSE converges to a specific limit at differentiable points.

At low SNR, all interior points are differentiable.

Abstract

In this short note, we consider models of optimal Bayesian inference of finite-rank tensor products. We add to the model a linear channel parametrized by $h$. We show that at every interior differentiable point $h$ of the free energy (associated with the model), the overlap concentrates at the gradient of the free energy and the minimum mean-square error converges to a related limit. In other words, the model is replica-symmetric at every differentiable point. At any signal-to-noise ratio, such points $h$ form a full-measure set (hence $h=0$ belongs to the closure of these points). For a sufficiently low signal-to-noise ratio, we show that every interior point is a differentiable point.