Cycle-based Cluster Variational Method for Direct and Inverse Inference

TL;DR

This paper introduces a cycle-based cluster variational method that improves belief propagation on pairwise Markov random fields, enhancing convergence and stability for both direct and inverse inference tasks.

Contribution

It proposes a systematic cycle-based region graph approach that reduces dual loops and redundant constraints, enabling efficient and stable inference in Markov random fields.

Findings

Effective in the Ising model context

Improves convergence and stability of belief propagation

Enables efficient inverse inference for coupling estimation

Abstract

We elaborate on the idea that loop corrections to belief propagation could be dealt with in a systematic way on pairwise Markov random fields, by using the elements of a cycle basis to define region in a generalized belief propagation setting. The region graph is specified in such a way as to avoid dual loops as much as possible, by discarding redundant Lagrange multipliers, in order to facilitate the convergence, while avoiding instabilities associated to minimal factor graph construction. We end up with a two-level algorithm, where a belief propagation algorithm is run alternatively at the level of each cycle and at the inter-region level. The inverse problem of finding the couplings of a Markov random field from empirical covariances can be addressed region wise. It turns out that this can be done efficiently in particular in the Ising context, where fixed point equations can be derived along with a one-parameter log likelihood function to minimize. Numerical experiments confirm the effectiveness of these considerations both for the direct and inverse MRF inference.