Passing Expectation Propagation Messages with Kernel Methods

TL;DR

This paper introduces a kernel-based message operator for expectation propagation that learns to directly produce outgoing messages from incoming messages, bypassing complex integral computations during inference.

Contribution

It presents a novel kernel-based approach to learn message operators for EP, enabling automated inference on various factors without explicit integral calculation.

Findings

Successfully learned message operators from data

Reduced computational complexity during inference

Applicable to diverse factor types

Abstract

We propose to learn a kernel-based message operator which takes as input all expectation propagation (EP) incoming messages to a factor node and produces an outgoing message. In ordinary EP, computing an outgoing message involves estimating a multivariate integral which may not have an analytic expression. Learning such an operator allows one to bypass the expensive computation of the integral during inference by directly mapping all incoming messages into an outgoing message. The operator can be learned from training data (examples of input and output messages) which allows automated inference to be made on any kind of factor that can be sampled.