Restricted Collapsed Draw: Accurate Sampling for Hierarchical Chinese Restaurant Process Hidden Markov Models

TL;DR

This paper introduces a restricted collapsed draw (RCD) sampler that enables accurate and efficient sampling for hierarchical Chinese restaurant process models, especially in complex scenarios like infinite Hidden Markov Models.

Contribution

The paper presents a novel RCD sampler that overcomes combinatorial challenges in hierarchical Dirichlet process models, improving sampling accuracy and efficiency.

Findings

RCD sampler achieves more accurate sampling in complex hierarchical models.

Developed advanced iHMM sampling algorithms based on RCD, outperforming traditional methods.

Experiments demonstrate improved performance and efficiency of the proposed methods.

Abstract

We propose a restricted collapsed draw (RCD) sampler, a general Markov chain Monte Carlo sampler of simultaneous draws from a hierarchical Chinese restaurant process (HCRP) with restriction. Models that require simultaneous draws from a hierarchical Dirichlet process with restriction, such as infinite Hidden markov models (iHMM), were difficult to enjoy benefits of \markerg{the} HCRP due to combinatorial explosion in calculating distributions of coupled draws. By constructing a proposal of seating arrangements (partitioning) and stochastically accepts the proposal by the Metropolis-Hastings algorithm, the RCD sampler makes accurate sampling for complex combination of draws while retaining efficiency of HCRP representation. Based on the RCD sampler, we developed a series of sophisticated sampling algorithms for iHMMs, including blocked Gibbs sampling, beam sampling, and split-merge sampling, that outperformed conventional iHMM samplers in experiments