On the Convexity and Reliability of the Bethe Free Energy Approximation

TL;DR

This paper analyzes the conditions under which the Bethe free energy approximation is reliable, focusing on convexity properties and introducing practical methods to verify and optimize it.

Contribution

It provides theoretical conditions based on the Bethe Hessian for assessing approximation reliability and introduces BETHE-MIN, a new optimization algorithm.

Findings

Convexity of the Bethe free energy correlates with approximation accuracy.

Two sufficient conditions for convexity are derived based on the Bethe Hessian matrix.

BETHE-MIN efficiently finds minima of the Bethe free energy.

Abstract

The Bethe free energy approximation provides an effective way for relaxing NP-hard problems of probabilistic inference. However, its accuracy depends on the model parameters and particularly degrades if a phase transition in the model occurs. In this work, we analyze when the Bethe approximation is reliable and how this can be verified. We argue and show by experiment that it is mostly accurate if it is convex on a submanifold of its domain, the 'Bethe box'. For verifying its convexity, we derive two sufficient conditions that are based on the definiteness properties of the Bethe Hessian matrix: the first uses the concept of diagonal dominance, and the second decomposes the Bethe Hessian matrix into a sum of sparse matrices and characterizes the definiteness properties of the individual matrices in that sum. These theoretical results provide a simple way to estimate the critical phase transition temperature of a model. As a practical contribution we propose $\texttt{BETHE-MIN}$, a projected quasi-Newton method to efficiently find a minimum of the Bethe free energy.