Dirichlet Posterior Sampling with Truncated Multinomial Likelihoods

TL;DR

This paper introduces a fast, scalable data augmentation algorithm for sampling from Dirichlet posteriors with truncated multinomial likelihoods, improving inference in hierarchical Bayesian models.

Contribution

It presents a novel, efficient sampling method for Dirichlet posteriors with truncated likelihoods, enhancing Gibbs sampling in hierarchical Bayesian models.

Findings

Algorithm outperforms Metropolis-Hastings in speed and mixing.

Scalable to high-dimensional problems.

Applicable to hierarchical Bayesian inference.

Abstract

We consider the problem of drawing samples from posterior distributions formed under a Dirichlet prior and a truncated multinomial likelihood, by which we mean a Multinomial likelihood function where we condition on one or more counts being zero a priori. Sampling this posterior distribution is of interest in inference algorithms for hierarchical Bayesian models based on the Dirichlet distribution or the Dirichlet process, particularly Gibbs sampling algorithms for the Hierarchical Dirichlet Process Hidden Semi-Markov Model. We provide a data augmentation sampling algorithm that is easy to implement, fast both to mix and to execute, and easily scalable to many dimensions. We demonstrate the algorithm's advantages over a generic Metropolis-Hastings sampling algorithm in several numerical experiments.