Topic Modelling Black Box Optimization

TL;DR

This paper compares different black-box optimization methods for selecting the number of topics in LDA, demonstrating that learned optimizers like SABBO and PABBO are more efficient than traditional evolutionary algorithms.

Contribution

It introduces a formulation of topic number selection as a black-box optimization problem and evaluates four optimization strategies, highlighting the efficiency of learned approaches.

Findings

Amortized optimizers reach near-optimal solutions with fewer evaluations.

SABBO identifies good topic numbers after nearly one evaluation.

Learned optimizers outperform evolutionary methods in efficiency.

Abstract

Choosing the number of topics $T$ in Latent Dirichlet Allocation (LDA) is a key design decision that strongly affects both the statistical fit and interpretability of topic models. In this work, we formulate the selection of $T$ as a discrete black-box optimization problem, where each function evaluation corresponds to training an LDA model and measuring its validation perplexity. Under a fixed evaluation budget, we compare four families of optimizers: two hand-designed evolutionary methods - Genetic Algorithm (GA) and Evolution Strategy (ES) - and two learned, amortized approaches, Preferential Amortized Black-Box Optimization (PABBO) and Sharpness-Aware Black-Box Optimization (SABBO). Our experiments show that, while GA, ES, PABBO, and SABBO eventually reach a similar band of final perplexity, the amortized optimizers are substantially more sample- and time-efficient. SABBO typically identifies a near-optimal topic number after essentially a single evaluation, and PABBO finds competitive configurations within a few evaluations, whereas GA and ES require almost the full budget to approach the same region.