Fast sampling and model selection for Bayesian mixture models

TL;DR

This paper introduces a rejection-free Monte Carlo algorithm for Bayesian mixture models that improves sampling efficiency by directly estimating the marginal posterior over component assignments, outperforming traditional Gibbs sampling.

Contribution

The paper presents a novel rejection-free Monte Carlo method for sampling from the marginal posterior in Bayesian mixture models, enhancing mixing times and efficiency.

Findings

Outperforms Gibbs sampling in mixing times

Effective for Gaussian, Poisson, and categorical models

Achieves significant speedups in typical applications

Abstract

We study Bayesian estimation of mixture models and argue in favor of fitting the marginal posterior distribution over component assignments directly, rather than Gibbs sampling from the joint posterior on components and parameters as is commonly done. Some previous authors have found the former approach to have slow mixing, but we show that, implemented correctly, it can achieve excellent performance. In particular, we describe a new Monte Carlo algorithm for sampling from the marginal posterior of a general integrable mixture that makes use of rejection-free sampling from the prior over component assignments to achieve excellent mixing times in typical applications, outperforming standard Gibbs sampling, in some cases by a wide margin. We demonstrate the approach with a selection of applications to Gaussian, Poisson, and categorical models.