Natural Gradient Variational Inference with Gaussian Mixture Models

TL;DR

This paper introduces a set of update rules for natural gradient variational inference using Gaussian mixture models, enabling efficient and parallelizable approximation of complex posterior distributions in Bayesian inference.

Contribution

It proposes novel update rules for natural gradient VI with Gaussian mixtures, allowing independent and parallel updates of mixture components.

Findings

Enables parallel computation of mixture components.

Improves approximation flexibility for complex posteriors.

Facilitates scalable Bayesian inference.

Abstract

Bayesian methods estimate a measure of uncertainty by using the posterior distribution. One source of difficulty in these methods is the computation of the normalizing constant. Calculating exact posterior is generally intractable and we usually approximate it. Variational Inference (VI) methods approximate the posterior with a distribution usually chosen from a simple family using optimization. The main contribution of this work is described is a set of update rules for natural gradient variational inference with mixture of Gaussians, which can be run independently for each of the mixture components, potentially in parallel.