Approximating the Bethe partition function

TL;DR

This paper develops improved algorithms for approximating the Bethe partition function in Markov Random Fields, providing a fully polynomial-time approximation scheme for attractive models and connecting the problem to MAP inference in general models.

Contribution

The authors introduce a new derivative-based approach that outperforms previous algorithms and extends to general models, establishing a link between Bethe approximation and MAP inference.

Findings

FPTAS for attractive models without degree restrictions

Bethe approximation performs well even when BP fails to converge

Approximation reduces to MAP inference in general models

Abstract

When belief propagation (BP) converges, it does so to a stationary point of the Bethe free energy $F$, and is often strikingly accurate. However, it may converge only to a local optimum or may not converge at all. An algorithm was recently introduced for attractive binary pairwise MRFs which is guaranteed to return an $\epsilon$-approximation to the global minimum of $F$ in polynomial time provided the maximum degree $\Delta=O(\log n)$, where $n$ is the number of variables. Here we significantly improve this algorithm and derive several results including a new approach based on analyzing first derivatives of $F$, which leads to performance that is typically far superior and yields a fully polynomial-time approximation scheme (FPTAS) for attractive models without any degree restriction. Further, the method applies to general (non-attractive) models, though with no polynomial time guarantee in this case, leading to the important result that approximating $\log$ of the Bethe partition function, $\log Z_B=-\min F$, for a general model to additive $\epsilon$-accuracy may be reduced to a discrete MAP inference problem. We explore an application to predicting equipment failure on an urban power network and demonstrate that the Bethe approximation can perform well even when BP fails to converge.