Borrowing strengh in hierarchical Bayes: Posterior concentration of the Dirichlet base measure

TL;DR

This paper analyzes how hierarchical Dirichlet processes enable more efficient inference by borrowing strength across groups, demonstrating improved convergence rates in estimating base measures with theoretical guarantees.

Contribution

It establishes convergence rates for hierarchical Dirichlet processes in Wasserstein distances and highlights the significant efficiency gains from hierarchical modeling.

Findings

Hierarchical Dirichlet processes improve convergence rates.

Borrowing strength can lead to parametric rate of convergence.

Theoretical tools include transportation distances and geometric properties of Dirichlet measures.

Abstract

This paper studies posterior concentration behavior of the base probability measure of a Dirichlet measure, given observations associated with the sampled Dirichlet processes, as the number of observations tends to infinity. The base measure itself is endowed with another Dirichlet prior, a construction known as the hierarchical Dirichlet processes (Teh et al. [J. Amer. Statist. Assoc. 101 (2006) 1566-1581]). Convergence rates are established in transportation distances (i.e., Wasserstein metrics) under various conditions on the geometry of the support of the true base measure. As a consequence of the theory, we demonstrate the benefit of "borrowing strength" in the inference of multiple groups of data - a powerful insight often invoked to motivate hierarchical modeling. In certain settings, the gain in efficiency due to the latent hierarchy can be dramatic, improving from a standard nonparametric rate to a parametric rate of convergence. Tools developed include transportation distances for nonparametric Bayesian hierarchies of random measures, the existence of tests for Dirichlet measures, and geometric properties of the support of Dirichlet measures.