Tree-Guided MCMC Inference for Normalized Random Measure Mixture Models

TL;DR

This paper introduces a hybrid inference algorithm called tree-guided MCMC (tgMCMC) for normalized random measure mixture models, combining the efficiency of IBHC with the convergence guarantees of MCMC, leading to faster and more reliable posterior inference.

Contribution

The paper proposes a novel hybrid inference method that integrates IBHC and MCMC for NRMM models, ensuring convergence and improving inference speed.

Findings

tgMCMC converges reliably due to MCMC guarantees.

The method outperforms pure IBHC and MCMC in experiments.

Demonstrated effectiveness on synthetic and real datasets.

Abstract

Normalized random measures (NRMs) provide a broad class of discrete random measures that are often used as priors for Bayesian nonparametric models. Dirichlet process is a well-known example of NRMs. Most of posterior inference methods for NRM mixture models rely on MCMC methods since they are easy to implement and their convergence is well studied. However, MCMC often suffers from slow convergence when the acceptance rate is low. Tree-based inference is an alternative deterministic posterior inference method, where Bayesian hierarchical clustering (BHC) or incremental Bayesian hierarchical clustering (IBHC) have been developed for DP or NRM mixture (NRMM) models, respectively. Although IBHC is a promising method for posterior inference for NRMM models due to its efficiency and applicability to online inference, its convergence is not guaranteed since it uses heuristics that simply selects the best solution after multiple trials are made. In this paper, we present a hybrid inference algorithm for NRMM models, which combines the merits of both MCMC and IBHC. Trees built by IBHC outlines partitions of data, which guides Metropolis-Hastings procedure to employ appropriate proposals. Inheriting the nature of MCMC, our tree-guided MCMC (tgMCMC) is guaranteed to converge, and enjoys the fast convergence thanks to the effective proposals guided by trees. Experiments on both synthetic and real-world datasets demonstrate the benefit of our method.