Beta processes, stick-breaking, and power laws

TL;DR

This paper introduces a new stick-breaking representation for the beta process, generalizes it with three parameters, explores power-law behaviors, and develops an inference algorithm with experimental validation.

Contribution

It derives a novel stick-breaking form for the beta process, generalizes it with three parameters, and applies it to power-law modeling and inference in feature models.

Findings

Derived a stick-breaking representation from the beta process as a completely random measure.

Proposed a three-parameter generalization of the beta process.

Developed an inference algorithm and demonstrated its effectiveness in experiments.

Abstract

The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.