Bayesian Inference for Non-Conjugate Distance Dependent Chinese Restaurant Process Models

TL;DR

This paper develops a reversible jump MCMC framework for Bayesian inference in non-conjugate distance dependent Chinese Restaurant Process models, enabling flexible clustering with covariate information.

Contribution

It introduces a novel RJMCMC approach with multiple proposal strategies for non-conjugate ddCRP models, improving inference in complex clustering scenarios.

Findings

Moment-matched proposals outperform prior-based proposals in posterior sampling.

The framework effectively handles both discrete and continuous observation models.

Application to Old Faithful data demonstrates practical utility.

Abstract

The distance dependent Chinese Restaurant Process (ddCRP) provides a flexible prior distribution for clustering observations, incorporating covariate information through pairwise distances and accommodating a rich variety of cluster structures. When cluster parameters are conjugate to the likelihood, Bayesian inference is straightforward. In the non-conjugate setting, however, inference becomes substantially more challenging due to the trans-dimensional parameter spaces that arise as cluster assignments change. We develop a reversible jump Markov chain Monte Carlo (RJMCMC) framework to address this challenge, targeting the dimension-changing nature of cluster parameter vectors when observation assignments are updated. We introduce and compare several proposal strategies for birth and death moves, including prior-based, independence, and data-driven moment-matching proposals that target regions of high posterior density. For fixed-dimensional moves, we propose a posterior resampling strategy that improves acceptance rates while maintaining computational efficiency. Through a simulation study and an application to Old Faithful eruption durations, we demonstrate moment-matched proposals offer a principled, data-driven alternative to prior-based proposals. The resulting methodology provides a general RJMCMC framework for ddCRP models with non-conjugate likelihoods, demonstrated here on both discrete and continuous observation models.