Evidence estimation in finite and infinite mixture models and applications

TL;DR

This paper reviews and introduces Monte Carlo methods for estimating model evidence in mixture models, including novel approaches, and discusses their applications in model selection and comparison.

Contribution

It presents new Monte Carlo techniques based on Geyer, Chib, and SMC methods for evidence estimation in finite and infinite mixture models, with theoretical consistency results.

Findings

Bayes factor provides a consistent method for selecting the number of components.

Established the consistency of Bayes factor when comparing finite mixtures to Dirichlet Process Mixtures.

Reviewed and proposed Monte Carlo techniques for evidence estimation in mixture models.

Abstract

Estimating the model evidence - or mariginal likelihood of the data - is a notoriously difficult task for finite and infinite mixture models and we reexamine here different Monte Carlo techniques advocated in the recent literature, as well as novel approaches based on Geyer (1994) reverse logistic regression technique, Chib (1995) algorithm, and Sequential Monte Carlo (SMC). Applications are numerous. In particular, testing for the number of components in a finite mixture model or against the fit of a finite mixture model for a given dataset has long been and still is an issue of much interest, albeit yet missing a fully satisfactory resolution. Using a Bayes factor to find the right number of components K in a finite mixture model is known to provide a consistent procedure. We furthermore establish the consistence of the Bayes factor when comparing a parametric family of finite mixtures against the nonparametric 'strongly identifiable' Dirichlet Process Mixture (DPM) model.