Belief Propagation Converges to Gaussian Distributions in Sparsely-Connected Factor Graphs

TL;DR

This paper provides a theoretical proof that belief propagation beliefs converge to Gaussian distributions in sparsely-connected factor graphs, supported by experiments in stereo depth estimation.

Contribution

It offers a mathematical guarantee for Gaussian convergence in loopy factor graphs under specific assumptions, enhancing understanding of GBP's effectiveness.

Findings

Beliefs become increasingly Gaussian after few BP iterations

Theoretical proof based on the Central Limit Theorem

Experimental validation in stereo depth estimation

Abstract

Belief Propagation (BP) is a powerful algorithm for distributed inference in probabilistic graphical models, however it quickly becomes infeasible for practical compute and memory budgets. Many efficient, non-parametric forms of BP have been developed, but the most popular is Gaussian Belief Propagation (GBP), a variant that assumes all distributions are locally Gaussian. GBP is widely used due to its efficiency and empirically strong performance in applications like computer vision or sensor networks - even when modelling non-Gaussian problems. In this paper, we seek to provide a theoretical guarantee for when Gaussian approximations are valid in highly non-Gaussian, sparsely-connected factor graphs performing BP (common in spatial AI). We leverage the Central Limit Theorem (CLT) to prove mathematically that variables' beliefs under BP converge to a Gaussian distribution in complex, loopy factor graphs obeying our 4 key assumptions. We then confirm experimentally that variable beliefs become increasingly Gaussian after just a few BP iterations in a stereo depth estimation task.