Kernel Belief Propagation

TL;DR

Kernel Belief Propagation (KBP) is a flexible, nonparametric extension of belief propagation that operates in a reproducing kernel Hilbert space, enabling it to handle complex, unknown, or implicit variable relations across diverse domains.

Contribution

The paper introduces KBP, a nonparametric belief propagation method using RKHS, allowing for flexible, data-driven message passing without parametric assumptions.

Findings

KBP outperforms classical and nonparametric methods in speed and accuracy.

KBP is applicable to diverse data types like images, strings, and groups.

KBP provides significant computational advantages in practical tasks.

Abstract

We propose a nonparametric generalization of belief propagation, Kernel Belief Propagation (KBP), for pairwise Markov random fields. Messages are represented as functions in a reproducing kernel Hilbert space (RKHS), and message updates are simple linear operations in the RKHS. KBP makes none of the assumptions commonly required in classical BP algorithms: the variables need not arise from a finite domain or a Gaussian distribution, nor must their relations take any particular parametric form. Rather, the relations between variables are represented implicitly, and are learned nonparametrically from training data. KBP has the advantage that it may be used on any domain where kernels are defined (Rd, strings, groups), even where explicit parametric models are not known, or closed form expressions for the BP updates do not exist. The computational cost of message updates in KBP is polynomial in the training data size. We also propose a constant time approximate message update procedure by representing messages using a small number of basis functions. In experiments, we apply KBP to image denoising, depth prediction from still images, and protein configuration prediction: KBP is faster than competing classical and nonparametric approaches (by orders of magnitude, in some cases), while providing significantly more accurate results.