The DLR Hierarchy of Approximate Inference

TL;DR

This paper introduces a hierarchical framework for approximate inference based on DLR equations, unifying existing algorithms and proposing new ones, with analysis and empirical comparison on spin-glass problems.

Contribution

It presents a novel hierarchy for approximate inference that encompasses and extends existing algorithms, linking them through the DLR equations and exploring their properties.

Findings

Higher hierarchy algorithms yield more accurate results when converged.

Algorithms higher in the hierarchy tend to be less stable.

Connections between approximate algorithms and Gibbs sampling are established.

Abstract

We propose a hierarchy for approximate inference based on the Dobrushin, Lanford, Ruelle (DLR) equations. This hierarchy includes existing algorithms, such as belief propagation, and also motivates novel algorithms such as factorized neighbors (FN) algorithms and variants of mean field (MF) algorithms. In particular, we show that extrema of the Bethe free energy correspond to approximate solutions of the DLR equations. In addition, we demonstrate a close connection between these approximate algorithms and Gibbs sampling. Finally, we compare and contrast various of the algorithms in the DLR hierarchy on spin-glass problems. The experiments show that algorithms higher up in the hierarchy give more accurate results when they converge but tend to be less stable.