Stochastic Variational Inference

TL;DR

This paper introduces stochastic variational inference, a scalable method for approximating posterior distributions in large probabilistic models, demonstrated on extensive text corpora and outperforming traditional methods.

Contribution

It presents a new stochastic variational inference algorithm applicable to a wide range of probabilistic models, enabling analysis of massive datasets.

Findings

Handles datasets with millions of documents efficiently.

Outperforms traditional variational inference in speed and accuracy.

Bayesian nonparametric models outperform parametric ones.

Abstract

We develop stochastic variational inference, a scalable algorithm for approximating posterior distributions. We develop this technique for a large class of probabilistic models and we demonstrate it with two probabilistic topic models, latent Dirichlet allocation and the hierarchical Dirichlet process topic model. Using stochastic variational inference, we analyze several large collections of documents: 300K articles from Nature, 1.8M articles from The New York Times, and 3.8M articles from Wikipedia. Stochastic inference can easily handle data sets of this size and outperforms traditional variational inference, which can only handle a smaller subset. (We also show that the Bayesian nonparametric topic model outperforms its parametric counterpart.) Stochastic variational inference lets us apply complex Bayesian models to massive data sets.