Derivation of the AMP equations from belief propagation for the $\ell_2$ minimisation problem

TL;DR

This paper derives the AMP equations from belief propagation for the $ ext{l}_2$ minimization problem, connecting statistical mechanics, probabilistic inference, and optimization in high-dimensional linear systems.

Contribution

It rigorously shows that belief propagation means asymptotically satisfy the AMP equations for the $ ext{l}_2$ minimization problem with random matrices.

Findings

AMP equations derived from belief propagation for $ ext{l}_2$ minimization.

Asymptotic equivalence of belief propagation means and AMP in the simplest case.

Connection between statistical mechanics and high-dimensional optimization.

Abstract

We consider the $\ell_p$-minimisation, which consists of finding the vector $x\in\mathbb{R}^N$ which minimises $\|x\|_p$ subject to the linear constraint $y=Ax$, where $y\in\mathbb{R}^m$ is given and $A$ is a $m\times N$ random matrix with i.i.d. sub-Gaussian centred entries ($m<N$). This can be viewed as the zero temperature version of a statistical mechanics problem, in which one introduces a suitable Gibbs measure on $\mathbb{R}^N$. To such a Gibbs measure there are associated belief propagation equations. We prove in the easiest case $p=2$ that the means of the distributions obtained by the belief propagation iteration satisfy asymptotically the approximate message passing equations.