$\alpha$ Belief Propagation as Fully Factorized Approximation

TL;DR

This paper introduces $oldsymbol{ extalpha}$-belief propagation ($ extalpha$-BP), an interpretable algorithm derived from localized $oldsymbol{ extalpha}$-divergence minimization, which improves MAP inference performance in loopy and fully-connected graphs.

Contribution

It proposes $ extalpha$-BP as a novel, interpretable belief propagation method based on $ extalpha$-divergence minimization, enhancing inference in complex graphs.

Findings

$ extalpha$-BP outperforms traditional loopy BP in MAP inference tasks.

The algorithm is effective in fully-connected graphs.

$ extalpha$-BP provides a better understanding of belief propagation solutions.

Abstract

Belief propagation (BP) can do exact inference in loop-free graphs, but its performance could be poor in graphs with loops, and the understanding of its solution is limited. This work gives an interpretable belief propagation rule that is actually minimization of a localized $\alpha$-divergence. We term this algorithm as $\alpha$ belief propagation ($\alpha$-BP). The performance of $\alpha$-BP is tested in MAP (maximum a posterior) inference problems, where $\alpha$-BP can outperform (loopy) BP by a significant margin even in fully-connected graphs.