Primal View on Belief Propagation

TL;DR

This paper reveals that belief propagation (BP) can be understood as seeking stationary points of a single, unconstrained function related to local log-partition functions, providing a primal perspective on BP's behavior.

Contribution

It establishes that BP's fixed points correspond to stationary points of a single function, clarifying the primal nature of BP beyond the dual formulation.

Findings

BP fixed points are stationary points of a single function.

This function is a linear combination of local log-partition functions.

Provides a primal view of belief propagation, linking it to a single optimization landscape.

Abstract

It is known that fixed points of loopy belief propagation (BP) correspond to stationary points of the Bethe variational problem, where we minimize the Bethe free energy subject to normalization and marginalization constraints. Unfortunately, this does not entirely explain BP because BP is a dual rather than primal algorithm to solve the Bethe variational problem -- beliefs are infeasible before convergence. Thus, we have no better understanding of BP than as an algorithm to seek for a common zero of a system of non-linear functions, not explicitly related to each other. In this theoretical paper, we show that these functions are in fact explicitly related -- they are the partial derivatives of a single function of reparameterizations. That means, BP seeks for a stationary point of a single function, without any constraints. This function has a very natural form: it is a linear combination of local log-partition functions, exactly as the Bethe entropy is the same linear combination of local entropies.