Connection between Annealed Free Energy and Belief Propagation on Random Factor Graph Ensembles

TL;DR

This paper establishes a connection between annealed free energy and Bethe free energy for various random factor graph ensembles, providing a new derivation of the replica symmetric solution.

Contribution

It generalizes the relationship between annealed free energy and Bethe free energy to all random regular, irregular, and Poisson ensembles, linking them through belief propagation equations.

Findings

Annealed free energy equals Bethe free energy under replica symmetric assumptions for regular ensembles.

The stationary conditions of the annealed free energy maximization coincide with belief propagation equations.

Provides a simple derivation of the replica symmetric solution for factor graph models.

Abstract

Recently, Vontobel showed the relationship between Bethe free energy and annealed free energy for protograph factor graph ensembles. In this paper, annealed free energy of any random regular, irregular and Poisson factor graph ensembles are connected to Bethe free energy. The annealed free energy is expressed as the solution of maximization problem whose stationary condition equations coincide with equations of belief propagation since the contribution to partition function of particular type of variable and factor nodes has similar form of minus Bethe free energy. It gives simple derivation of replica symmetric solution. As consequence, it is shown that on replica symmetric ansatz, replica symmetric solution and annealed free energy are equal for regular ensemble.