On Expectation Propagation and the Probabilistic Editor in some simple mixture problems

TL;DR

This paper investigates the use of Expectation Propagation for mixture problems, showing it can produce more accurate posterior variances than Variational Bayes, leading to more reliable interval estimates.

Contribution

It demonstrates that Expectation Propagation can yield asymptotically correct variances in certain mixture problems, improving upon traditional variational methods.

Findings

Expectation Propagation provides more accurate variance estimates.

Variational Bayes tends to underestimate variances.

EP-based methods offer reliable interval estimates.

Abstract

As for other latent-variable problems, exact Bayesian analysis is typically not practicable for mixture problems and approximate methods have been developed. Variational Bayes tends to produce approximate posterior distributions for parameters that are too tightly concentrated in having variances that are too small. The paper identifies a few mixture problems in which Expectation Propagation and variations thereof lead to approximate posterior distributions that asymptotically exhibit `correct' variances and therefore stand to provide reliable interval estimates for the unknown parameter or parameters.