Block Belief Propagation for Parameter Learning in Markov Random Fields

TL;DR

This paper introduces block belief propagation learning (BBPL), a scalable method for training Markov random fields that reduces inference complexity by using block-coordinate updates, maintaining convergence to the same solution as full inference.

Contribution

The paper proposes BBPL, a novel scalable learning algorithm for Markov random fields that avoids full inference, with proven convergence and empirical scalability benefits.

Findings

BBPL converges to the same solution as full inference methods.

BBPL significantly improves scalability over standard training methods.

Empirical results demonstrate BBPL's efficiency on large graphical models.

Abstract

Traditional learning methods for training Markov random fields require doing inference over all variables to compute the likelihood gradient. The iteration complexity for those methods therefore scales with the size of the graphical models. In this paper, we propose \emph{block belief propagation learning} (BBPL), which uses block-coordinate updates of approximate marginals to compute approximate gradients, removing the need to compute inference on the entire graphical model. Thus, the iteration complexity of BBPL does not scale with the size of the graphs. We prove that the method converges to the same solution as that obtained by using full inference per iteration, despite these approximations, and we empirically demonstrate its scalability improvements over standard training methods.