Markov Chains Approximate Message Passing

TL;DR

This paper establishes a theoretical connection between Markov chain Monte Carlo algorithms and Approximate Message Passing (AMP) in Bayesian inference, demonstrating that Glauber dynamics can achieve Bayes-optimal recovery in the spiked Wigner model.

Contribution

It introduces the restricted Gaussian dynamics (RGD) as an auxiliary Markov chain to analyze AMP performance and links the convergence of Glauber dynamics to optimal inference results.

Findings

RGD reduces to a one-dimensional recursion similar to AMP.

RGD rapidly converges to a fixed point from a warm start.

Conditional on mixing conjectures, the phase transition for inference is recovered.

Abstract

Markov chain Monte Carlo algorithms have long been observed to obtain near-optimal performance in various Bayesian inference settings. However, developing a supporting theory that makes these studies rigorous has proved challenging. In this paper, we study the classical spiked Wigner inference problem, where one aims to recover a planted Boolean spike from a noisy matrix measurement. We relate the recovery performance of Glauber dynamics on the annealed posterior to the performance of Approximate Message Passing (AMP), which is known to achieve Bayes-optimal performance. Our main results rely on the analysis of an auxiliary Markov chain called restricted Gaussian dynamics (RGD). Concretely, we establish the following results: 1. RGD can be reduced to an effective one-dimensional recursion which mirrors the evolution of the AMP iterates. 2. From a warm start, RGD rapidly converges to a fixed point in correlation space, which recovers Bayes-optimal performance when run on the posterior. 3. Conditioned on widely believed mixing results for the SK model, we recover the phase transition for non-trivial inference.