Hierarchical random measures without tables

TL;DR

This paper introduces a new framework for hierarchical random measures that eliminates the need for tables in the hierarchical Dirichlet process, enabling faster, exact sampling and extending to broader classes of models.

Contribution

It proposes a prior for the concentration parameter that yields a quasi-conjugate posterior and removes the computational complexity of managing tables, extending to normalized hierarchical random measures.

Findings

Developed a faster, exact sampling algorithm for hierarchical Dirichlet processes.

Extended the framework to a new class of normalized hierarchical random measures.

Demonstrated improved interpretability and computational efficiency.

Abstract

The hierarchical Dirichlet process is the cornerstone of Bayesian nonparametric multilevel models. Its generative model can be described through a set of latent variables, commonly referred to as tables within the popular restaurant franchise metaphor. The latent tables simplify the expression of the posterior and allow for the implementation of a Gibbs sampling algorithm to approximately draw samples from it. However, managing their assignments can become computationally expensive, especially as the size of the dataset and of the number of levels increase. In this work, we identify a prior for the concentration parameter of the hierarchical Dirichlet process that (i) induces a quasi-conjugate posterior distribution, and (ii) removes the need of tables, bringing to more interpretable expressions for the posterior, with both a faster and an exact algorithm to sample from it. Remarkably, this construction extends beyond the Dirichlet process, leading to a new framework for defining normalized hierarchical random measures and a new class of algorithms to sample from their posteriors. The key analytical tool is the independence of multivariate increments, that is, their representation as completely random vectors.