Managing sparsity, time, and quality of inference in topic models

TL;DR

This paper introduces FW, a flexible and efficient inference framework for probabilistic topic models that produces sparse representations, converges linearly, and allows trade-offs between sparsity, quality, and speed.

Contribution

The paper presents FW, a novel inference framework that is adaptable, fast, and capable of handling nonconjugate priors in topic models, improving efficiency and sparsity control.

Findings

FW demonstrates linear convergence in inference tasks.

The framework effectively balances sparsity, quality, and computational time.

Inference in nonconjugate models is made more efficient with FW.

Abstract

Inference is an integral part of probabilistic topic models, but is often non-trivial to derive an efficient algorithm for a specific model. It is even much more challenging when we want to find a fast inference algorithm which always yields sparse latent representations of documents. In this article, we introduce a simple framework for inference in probabilistic topic models, denoted by FW. This framework is general and flexible enough to be easily adapted to mixture models. It has a linear convergence rate, offers an easy way to incorporate prior knowledge, and provides us an easy way to directly trade off sparsity against quality and time. We demonstrate the goodness and flexibility of FW over existing inference methods by a number of tasks. Finally, we show how inference in topic models with nonconjugate priors can be done efficiently.