Posterior distributions for Hierarchical Spike and Slab Indian Buffet processes

TL;DR

This paper introduces hierarchical spike and slab Indian Buffet processes, providing explicit descriptions of their distributions, enabling exact sampling, and extending their applicability to various latent feature models.

Contribution

It develops hierarchical spike and slab IBP models, offering novel descriptions of their distributions and establishing connections to existing models for flexible latent feature allocation.

Findings

Explicit marginal, posterior, and predictive distributions provided.

Enables exact sampling and practical implementation.

Links to existing IBP models and potential applications in various domains.

Abstract

Bayesian nonparametric hierarchical priors are highly effective in providing flexible models for latent data structures exhibiting sharing of information between and across groups. Most prominent is the Hierarchical Dirichlet Process (HDP), and its subsequent variants, which model latent clustering between and across groups. The HDP, may be viewed as a more flexible extension of Latent Dirichlet Allocation models (LDA), and has been applied to, for example, topic modelling, natural language processing, and datasets arising in health-care. We focus on analogous latent feature allocation models, where the data structures correspond to multisets or unbounded sparse matrices. The fundamental development in this regard is the Hierarchical Indian Buffet process (HIBP), which utilizes a hierarchy of Beta processes over J groups, where each group generates binary random matrices, reflecting within group sharing of features, according to beta-Bernoulli IBP priors. To encompass HIBP versions of non-Bernoulli extensions of the IBP, we introduce hierarchical versions of general spike and slab IBP. We provide explicit novel descriptions of the marginal, posterior and predictive distributions of the HIBP and its generalizations which allow for exact sampling and simpler practical implementation. We highlight common structural properties of these processes and establish relationships to existing IBP type and related models arising in the literature. Examples of potential applications may involve topic models, Poisson factorization models, random count matrix priors and neural network models