On the use of Markov chain Monte Carlo methods for the sampling of mixture models

TL;DR

This paper analyzes asymptotic properties of MCMC algorithms for mixture models, introduces a new efficient FCC sampler, and compares its performance with existing methods both theoretically and numerically.

Contribution

The paper introduces the FCC sampler, a computationally efficient MCMC algorithm for mixture models, and provides a theoretical and empirical comparison with existing schemes.

Findings

FCC sampler reduces computational demand compared to Metropolised schemes.

The new algorithm has higher asymptotic variance but improved efficiency.

Theoretical analysis confirms the performance benefits of the FCC sampler.

Abstract

In this paper we study asymptotic properties of different data-augmentation-type Markov chain Monte Carlo algorithms sampling from mixture models comprising discrete as well as continuous random variables. Of particular interest to us is the situation where sampling from the conditional distribution of the continuous component given the discrete component is infeasible. In this context, we cast Carlin & Chib's pseudo-prior method into the framework of mixture models and discuss and compare different variants of this scheme. We propose a novel algorithm, the FCC sampler, which is less computationally demanding than any Metropolised Carlin & Chib-type algorithm. The significant gain of computational efficiency is however obtained at the cost of some asymptotic variance. The performance of the algorithm vis-\`a-vis alternative schemes is investigated theoretically, using some recent results obtained in [3] for inhomogeneous Markov chains evolving alternatingly according to two different reversible Markov transition kernels, as well as numerically.