Convergence Rates of Latent Topic Models Under Relaxed Identifiability Conditions

TL;DR

This paper establishes the convergence rate of the maximum likelihood estimator in latent Dirichlet allocation models, demonstrating it converges at a rate of n^{-1/4} under relaxed conditions, which is proven to be optimal.

Contribution

It generalizes previous results by proving the optimal n^{-1/4} convergence rate without strict assumptions on topic separability or document length.

Findings

MLE converges at n^{-1/4} rate in Wasserstein distance

Convergence rate holds without separability or non-degeneracy assumptions

The n^{-1/4} rate is proven to be optimal in the worst case

Abstract

In this paper we study the frequentist convergence rate for the Latent Dirichlet Allocation (Blei et al., 2003) topic models. We show that the maximum likelihood estimator converges to one of the finitely many equivalent parameters in Wasserstein's distance metric at a rate of $n^{-1/4}$ without assuming separability or non-degeneracy of the underlying topics and/or the existence of more than three words per document, thus generalizing the previous works of Anandkumar et al. (2012, 2014) from an information-theoretical perspective. We also show that the $n^{-1/4}$ convergence rate is optimal in the worst case.