Vectorized Bayesian Inference for Latent Dirichlet-Tree Allocation

TL;DR

This paper introduces Latent Dirichlet-Tree Allocation (LDTA), a flexible hierarchical topic model that generalizes LDA with tree-structured priors, and develops scalable, vectorized inference methods for it.

Contribution

It proposes LDTA, a novel hierarchical extension of LDA using Dirichlet-Trees, and develops fully vectorized inference algorithms for scalable, GPU-accelerated implementation.

Findings

Enables expressive hierarchical topic modeling.

Provides scalable, GPU-accelerated inference methods.

Maintains computational efficiency with increased modeling capacity.

Abstract

Latent Dirichlet Allocation (LDA) is a foundational model for discovering latent thematic structure in discrete data, but its Dirichlet prior cannot represent the rich correlations and hierarchical relationships often present among topics. We introduce the framework of Latent Dirichlet-Tree Allocation (LDTA), a generalization of LDA that replaces the Dirichlet prior with an arbitrary Dirichlet-Tree (DT) distribution. LDTA preserves LDA's generative structure but enables expressive, tree-structured priors over topic proportions. To perform inference, we develop universal mean-field variational inference and Expectation Propagation, providing tractable updates for all DT. We reveal the vectorized nature of the two inference methods through theoretical development, and perform fully vectorized, GPU-accelerated implementations. The resulting framework substantially expands the modeling capacity of LDA while maintaining scalability and computational efficiency.