Statistical Analysis of Loopy Belief Propagation in Random Fields

TL;DR

This paper introduces an analytical method to evaluate the average behavior of loopy belief propagation in random fields, applicable to general pairwise Markov random fields, with validation in Bayesian image restoration.

Contribution

It presents a novel analytical approach using the replica cluster variation method to compute quenched averages of LBP in complex MRFs, aligning well with numerical results.

Findings

Analytical evaluation of LBP averages matches numerical simulations.

Method applicable to diverse pairwise MRFs with different random field distributions.

Successful application to Bayesian image restoration demonstrating practical relevance.

Abstract

Loopy belief propagation (LBP), which is equivalent to the Bethe approximation in statistical mechanics, is a message-passing-type inference method that is widely used to analyze systems based on Markov random fields (MRFs). In this paper, we propose a message-passing-type method to analytically evaluate the quenched average of LBP in random fields by using the replica cluster variation method. The proposed analytical method is applicable to general pair-wise MRFs with random fields whose distributions differ from each other and can give the quenched averages of the Bethe free energies over random fields, which are consistent with numerical results. The order of its computational cost is equivalent to that of standard LBP. In the latter part of this paper, we describe the application of the proposed method to Bayesian image restoration, in which we observed that our theoretical results are in good agreement with the numerical results for natural images.