IID Sampling from Posterior Dirichlet Process Mixtures

TL;DR

This paper introduces a new method for generating independent and identically distributed samples from the posterior of Dirichlet process mixtures, addressing longstanding challenges in Bayesian nonparametrics.

Contribution

It develops a novel iid sampling methodology for Dirichlet process mixture posteriors, applicable to a broad class of models, improving sampling efficiency and accuracy.

Findings

Efficient iid sampling achieved in minutes for benchmark datasets.

Method successfully applied to enzyme, acidity, and galaxy datasets.

Parallel implementation enhances computational speed.

Abstract

The influence of Dirichlet process mixture is ubiquitous in the Bayesian nonparametrics literature. But sampling from its posterior distribution remains a challenge, despite the advent of various Markov chain Monte Carlo methods. The primary challenge is the infinite-dimensional setup, and even if the infinite-dimensional random measure is integrated out, high-dimensionality and discreteness still remain difficult issues to deal with. In this article, exploiting the key ideas proposed in Bhattacharya (2021b), we propose a novel methodology for drawing iid realizations from posteriors of Dirichlet process mixtures. We focus in particular on the more general and flexible model of Bhattacharya (2008), so that the methods developed here are simply applicable to the traditional Dirichlet process mixture. We illustrate our ideas on the well-known enzyme, acidity and the galaxy datasets, which are usually considered benchmark datasets for mixture applications. Generating 10, 000 iid realizations from the Dirichlet process mixture posterior of Bhattacharya (2008) given these datasets took 19 minutes, 8 minutes and 5 minutes, respectively, in our parallel implementation.