Expectation Propagation

TL;DR

Expectation Propagation is a unifying framework for various iterative approximation algorithms in Bayesian inference, connecting methods like variational inference, mean-field, and belief propagation, with extensions and applications.

Contribution

The paper introduces Expectation Propagation as a unifying approach, clarifies its relation to other methods, and discusses advanced topics and applications.

Findings

Expectation Propagation unifies multiple approximation algorithms.

Connections between Expectation Propagation and variational methods are established.

Extensions and corrections to Expectation Propagation improve inference accuracy.

Abstract

Variational inference is a powerful concept that underlies many iterative approximation algorithms; expectation propagation, mean-field methods and belief propagations were all central themes at the school that can be perceived from this unifying framework. The lectures of Manfred Opper introduce the archetypal example of Expectation Propagation, before establishing the connection with the other approximation methods. Corrections by expansion about the expectation propagation are then explained. Finally some advanced inference topics and applications are explored in the final sections.