Sampling from Dirichlet process mixture models with unknown concentration parameter: Mixing issues in large data implementations

TL;DR

This paper presents an advanced MCMC sampling method for Dirichlet process mixture models with unknown concentration parameters, addressing mixing issues and demonstrating effective inference on synthetic and real data.

Contribution

It introduces a novel Gibbs sampling algorithm combining slice and retrospective sampling, along with techniques for joint inference on the concentration parameter and label switching.

Findings

Demonstrates improved mixing in synthetic data

Shows effective inference on real-world data

Provides open-source C++ implementation

Abstract

We consider the question of Markov chain Monte Carlo sampling from a general stick-breaking Dirichlet process mixture model, with concentration parameter alpha. This paper introduces a Gibbs sampling algorithm that combines the slice sampling approach of Walker (2007) and the retrospective sampling approach of Papaspiliopoulos and Roberts (2008). Our general algorithm is implemented as efficient open source C++ software, available as an R package, and is based on a blocking strategy similar to that suggested by Papaspiliopoulos (2008) and implemented by Yau et al (2011). We discuss the difficulties of achieving good mixing in MCMC samplers of this nature and investigate sensitivity to initialisation. We additionally consider the challenges when an additional layer of hierarchy is added such that joint inference is to be made on alpha. We introduce a new label switching move and compute the marginal model posterior to help to surmount these difficulties. Our work is illustrated using a profile regression (Molitor et al, 2010) application, where we demonstrate good mixing behaviour for both synthetic and real examples.