Understanding the dynamics of message passing algorithms: a free probability heuristics

TL;DR

This paper applies free probability heuristics to analyze the dynamics of message passing algorithms in large dense systems, providing insights into their convergence and memory properties.

Contribution

It introduces a novel application of free probability theory to understand the behavior of inference algorithms with dense coupling matrices.

Findings

Recovered known properties like vanishing effective memories

Derived analytical convergence rates for the algorithms

Validated the approach on a toy Ising model

Abstract

We use freeness assumptions of random matrix theory to analyze the dynamical behavior of inference algorithms for probabilistic models with dense coupling matrices in the limit of large systems. For a toy Ising model, we are able to recover previous results such as the property of vanishing effective memories and the analytical convergence rate of the algorithm.