E-LDA: Toward Interpretable LDA Topic Models with Strong Guarantees in Logarithmic Parallel Time

TL;DR

This paper introduces a novel, provably guaranteed, combinatorial algorithm for LDA inference that is exponentially faster, interpretable, and suitable for causal inference, outperforming existing methods in quality and speed.

Contribution

It presents the first practical, non-gradient-based algorithm for LDA inference with provable guarantees and interpretability, achieving logarithmic parallel time complexity.

Findings

Algorithms converge to near-optimal posterior probabilities.

Solutions have higher semantic quality than existing methods.

Approach maintains independence assumptions for causal inference.

Abstract

In this paper, we provide the first practical algorithms with provable guarantees for the problem of inferring the topics assigned to each document in an LDA topic model. This is the primary inference problem for many applications of topic models in social science, data exploration, and causal inference settings. We obtain this result by showing a novel non-gradient-based, combinatorial approach to estimating topic models. This yields algorithms that converge to near-optimal posterior probability in logarithmic parallel computation time (adaptivity) -- exponentially faster than any known LDA algorithm. We also show that our approach can provide interpretability guarantees such that each learned topic is formally associated with a known keyword. Finally, we show that unlike alternatives, our approach can maintain the independence assumptions necessary to use the learned topic model for downstream causal inference methods that allow researchers to study topics as treatments. In terms of practical performance, our approach consistently returns solutions of higher semantic quality than solutions from state-of-the-art LDA algorithms, neural topic models, and LLM-based topic models across a diverse range of text datasets and evaluation parameters.