
TL;DR
This paper reveals that belief propagation (BP) can be understood as seeking stationary points of a single, unconstrained function related to local log-partition functions, providing a primal perspective on BP's behavior.
Contribution
It establishes that BP's fixed points correspond to stationary points of a single function, clarifying the primal nature of BP beyond the dual formulation.
Findings
BP fixed points are stationary points of a single function.
This function is a linear combination of local log-partition functions.
Provides a primal view of belief propagation, linking it to a single optimization landscape.
Abstract
It is known that fixed points of loopy belief propagation (BP) correspond to stationary points of the Bethe variational problem, where we minimize the Bethe free energy subject to normalization and marginalization constraints. Unfortunately, this does not entirely explain BP because BP is a dual rather than primal algorithm to solve the Bethe variational problem -- beliefs are infeasible before convergence. Thus, we have no better understanding of BP than as an algorithm to seek for a common zero of a system of non-linear functions, not explicitly related to each other. In this theoretical paper, we show that these functions are in fact explicitly related -- they are the partial derivatives of a single function of reparameterizations. That means, BP seeks for a stationary point of a single function, without any constraints. This function has a very natural form: it is a linear…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsError Correcting Code Techniques · Bayesian Modeling and Causal Inference · Algorithms and Data Compression
