A sequential algorithm for fast fitting of Dirichlet process mixture models

TL;DR

This paper introduces VSUGS, a variational Bayes extension of the SUGS algorithm, enabling faster and more accurate Bayesian inference for large Dirichlet process mixture models, with demonstrated improvements in density estimation and classification.

Contribution

It proposes a novel variational SUGS algorithm that softens data allocation, improving speed and accuracy over the original SUGS method for large-scale mixture modeling.

Findings

VSUGS outperforms SUGS in density estimation.

VSUGS provides better classification results.

Application to flow cytometry and SNP data shows improved modeling.

Abstract

In this article we propose an improvement on the sequential updating and greedy search (SUGS) algorithm Wang and Dunson for fast fitting of Dirichlet process mixture models. The SUGS algorithm provides a means for very fast approximate Bayesian inference for mixture data which is particularly of use when data sets are so large that many standard Markov chain Monte Carlo (MCMC) algorithms cannot be applied efficiently, or take a prohibitively long time to converge. In particular, these ideas are used to initially interrogate the data, and to refine models such that one can potentially apply exact data analysis later on. SUGS relies upon sequentially allocating data to clusters and proceeding with an update of the posterior on the subsequent allocations and parameters which assumes this allocation is correct. Our modification softens this approach, by providing a probability distribution over allocations, with a similar computational cost; this approach has an interpretation as a variational Bayes procedure and hence we term it variational SUGS (VSUGS). It is shown in simulated examples that VSUGS can out-perform, in terms of density estimation and classification, the original SUGS algorithm in many scenarios. In addition, we present a data analysis for flow cytometry data, and SNP data via a three-class dirichlet process mixture model illustrating the apparent improvement over SUGS.