On collapsed representation of hierarchical Completely Random Measures

TL;DR

This paper introduces an exact method for generating Poisson processes from hierarchical Completely Random Measures (CRMs) without instantiating infinite atoms, enabling efficient Bayesian modeling and inference.

Contribution

It derives the marginal distribution for hierarchical CRM-based Poisson processes and develops Gibbs sampling strategies, with an application to topic modeling using the sum of generalized gamma processes.

Findings

Exact Poisson process sampling from hierarchical CRM

Gibbs sampling strategies for hierarchical CRM models

Application to Bayesian topic modeling with power-law behavior

Abstract

The aim of the paper is to provide an exact approach for generating a Poisson process sampled from a hierarchical CRM, without having to instantiate the infinitely many atoms of the random measures. We use completely random measures~(CRM) and hierarchical CRM to define a prior for Poisson processes. We derive the marginal distribution of the resultant point process, when the underlying CRM is marginalized out. Using well known properties unique to Poisson processes, we were able to derive an exact approach for instantiating a Poisson process with a hierarchical CRM prior. Furthermore, we derive Gibbs sampling strategies for hierarchical CRM models based on Chinese restaurant franchise sampling scheme. As an example, we present the sum of generalized gamma process (SGGP), and show its application in topic-modelling. We show that one can determine the power-law behaviour of the topics and words in a Bayesian fashion, by defining a prior on the parameters of SGGP.