Latent nested nonparametric priors

TL;DR

This paper introduces a new class of latent nested nonparametric priors that improve dependence modeling in Bayesian nonparametrics, overcoming limitations of the nested Dirichlet process, with applications demonstrated on synthetic and real data.

Contribution

The paper proposes latent nested processes combining common and group-specific measures, providing a flexible dependence structure and a new inference method.

Findings

Latent nested processes generalize nested Dirichlet processes.

The new models capture a range of dependence from exchangeability to independence.

A Bayesian inference algorithm and a homogeneity test are developed.

Abstract

Discrete random structures are important tools in Bayesian nonparametrics and the resulting models have proven effective in density estimation, clustering, topic modeling and prediction, among others. In this paper, we consider nested processes and study the dependence structures they induce. Dependence ranges between homogeneity, corresponding to full exchangeability, and maximum heterogeneity, corresponding to (unconditional) independence across samples. The popular nested Dirichlet process is shown to degenerate to the fully exchangeable case when there are ties across samples at the observed or latent level. To overcome this drawback, inherent to nesting general discrete random measures, we introduce a novel class of latent nested processes. These are obtained by adding common and group-specific completely random measures and, then, normalising to yield dependent random probability measures. We provide results on the partition distributions induced by latent nested processes, and develop an Markov Chain Monte Carlo sampler for Bayesian inferences. A test for distributional homogeneity across groups is obtained as a by product. The results and their inferential implications are showcased on synthetic and real data.