Mixture models with a prior on the number of components

TL;DR

This paper demonstrates that mixture models with a prior on the number of components (MFMs) share many properties with Dirichlet process models, allowing similar inference methods to be applied.

Contribution

It shows that MFMs possess key properties of DPMs, enabling the use of established inference techniques for models with an unknown number of mixture components.

Findings

MFMs exhibit exchangeable partition distribution and restaurant process.

Inference methods for DPMs are applicable to MFMs.

Validated with simulated and high-dimensional gene expression data.

Abstract

A natural Bayesian approach for mixture models with an unknown number of components is to take the usual finite mixture model with Dirichlet weights, and put a prior on the number of components---that is, to use a mixture of finite mixtures (MFM). While inference in MFMs can be done with methods such as reversible jump Markov chain Monte Carlo, it is much more common to use Dirichlet process mixture (DPM) models because of the relative ease and generality with which DPM samplers can be applied. In this paper, we show that, in fact, many of the attractive mathematical properties of DPMs are also exhibited by MFMs---a simple exchangeable partition distribution, restaurant process, random measure representation, and in certain cases, a stick-breaking representation. Consequently, the powerful methods developed for inference in DPMs can be directly applied to MFMs as well. We illustrate with simulated and real data, including high-dimensional gene expression data.