Adaptive Variational Inference in Probabilistic Graphical Models: Beyond Bethe, Tree-Reweighted, and Convex Free Energies

TL;DR

This paper introduces adaptive variational inference methods for probabilistic graphical models that improve approximation quality by automatically adjusting to complex models, surpassing traditional Bethe and convex free energy approaches.

Contribution

It proposes novel adaptive free energy approximations that tailor to specific models, enhancing inference accuracy in complex, highly interactive probabilistic graphical models.

Findings

Adaptive methods outperform traditional approaches on complex models

Proposed approximations demonstrate improved accuracy in difficult problems

Methods automatically adjust to model parameters and entropy approximations

Abstract

Variational inference in probabilistic graphical models aims to approximate fundamental quantities such as marginal distributions and the partition function. Popular approaches are the Bethe approximation, tree-reweighted, and other types of convex free energies. These approximations are efficient but can fail if the model is complex and highly interactive. In this work, we analyze two classes of approximations that include the above methods as special cases: first, if the model parameters are changed; and second, if the entropy approximation is changed. We discuss benefits and drawbacks of either approach, and deduce from this analysis how a free energy approximation should ideally be constructed. Based on our observations, we propose approximations that automatically adapt to a given model and demonstrate their effectiveness for a range of difficult problems.