Bayesian Nonparametrics in Topic Modeling: A Brief Tutorial

TL;DR

This paper provides a tutorial on Bayesian nonparametric methods in topic modeling, comparing models like Hierarchical Dirichlet Processes and Indian Buffet Process with traditional parametric models such as LDA.

Contribution

It offers a concise overview of nonparametric Bayesian methods in topic modeling and discusses potential computational advantages using IBP.

Findings

Comparison between nonparametric models and LDA

Discussion on inference speed and flexibility

Potential of IBP for efficient solutions

Abstract

Using nonparametric methods has been increasingly explored in Bayesian hierarchical modeling as a way to increase model flexibility. Although the field shows a lot of promise, inference in many models, including Hierachical Dirichlet Processes (HDP), remain prohibitively slow. One promising path forward is to exploit the submodularity inherent in Indian Buffet Process (IBP) to derive near-optimal solutions in polynomial time. In this work, I will present a brief tutorial on Bayesian nonparametric methods, especially as they are applied to topic modeling. I will show a comparison between different non-parametric models and the current state-of-the-art parametric model, Latent Dirichlet Allocation (LDA).