$\alpha$ Belief Propagation for Approximate Inference

TL;DR

This paper introduces $oldsymbol{ extalpha}$-belief propagation ($ extalpha$-BP), a generalized message-passing algorithm for approximate inference in graphical models that extends standard BP and includes convergence analysis and practical validation.

Contribution

The paper derives an interpretable $ extalpha$-BP algorithm based on $ extalpha$-divergence minimization, generalizing standard BP and providing convergence conditions.

Findings

$ extalpha$-BP generalizes standard belief propagation.

Convergence conditions for $ extalpha$-BP are established.

Experimental results validate theoretical convergence and effectiveness.

Abstract

Belief propagation (BP) algorithm is a widely used message-passing method for inference in graphical models. BP on loop-free graphs converges in linear time. But for graphs with loops, BP's performance is uncertain, and the understanding of its solution is limited. To gain a better understanding of BP in general graphs, we derive an interpretable belief propagation algorithm that is motivated by minimization of a localized $\alpha$-divergence. We term this algorithm as $\alpha$ belief propagation ($\alpha$-BP). It turns out that $\alpha$-BP generalizes standard BP. In addition, this work studies the convergence properties of $\alpha$-BP. We prove and offer the convergence conditions for $\alpha$-BP. Experimental simulations on random graphs validate our theoretical results. The application of $\alpha$-BP to practical problems is also demonstrated.