Analysis of Random Sequential Message Passing Algorithms for Approximate Inference

TL;DR

This paper provides an exact analysis of a random sequential message passing algorithm for approximate inference in large Gaussian models, revealing conditions for convergence and stability in mismatched scenarios.

Contribution

It introduces a dynamical functional approach to derive exact mean-field equations and identifies the stability boundary matching the AT condition.

Findings

Exact dynamical mean-field equations derived

Identified parameter regimes where the algorithm fails to converge

Stability boundary coincides with the AT condition

Abstract

We analyze the dynamics of a random sequential message passing algorithm for approximate inference with large Gaussian latent variable models in a student-teacher scenario. To model nontrivial dependencies between the latent variables, we assume random covariance matrices drawn from rotation invariant ensembles. Moreover, we consider a model mismatching setting, where the teacher model and the one used by the student may be different. By means of dynamical functional approach, we obtain exact dynamical mean-field equations characterizing the dynamics of the inference algorithm. We also derive a range of model parameters for which the sequential algorithm does not converge. The boundary of this parameter range coincides with the de Almeida Thouless (AT) stability condition of the replica symmetric ansatz for the static probabilistic model.