Structured Stochastic Variational Inference

TL;DR

This paper introduces a relaxation of the mean-field assumption in stochastic variational inference, allowing for dependencies between variables, which improves the accuracy and robustness of posterior approximations in large datasets.

Contribution

It proposes a new method to relax the mean-field approximation in stochastic variational inference, enabling more accurate posterior estimates with dependencies.

Findings

Improved posterior approximation accuracy.

Reduced bias and sensitivity to local optima.

Enhanced robustness to hyperparameters.

Abstract

Stochastic variational inference makes it possible to approximate posterior distributions induced by large datasets quickly using stochastic optimization. The algorithm relies on the use of fully factorized variational distributions. However, this "mean-field" independence approximation limits the fidelity of the posterior approximation, and introduces local optima. We show how to relax the mean-field approximation to allow arbitrary dependencies between global parameters and local hidden variables, producing better parameter estimates by reducing bias, sensitivity to local optima, and sensitivity to hyperparameters.