A New Spectral Method for Latent Variable Models

TL;DR

This paper introduces a spectral decomposition-based algorithm for unsupervised learning of latent variable models, demonstrating robustness and efficiency in parameter estimation for text mining applications like topic models and LDA.

Contribution

It presents a novel spectral method that improves parameter learning in latent variable models, with practical algorithms for text mining models such as single topic and LDA.

Findings

Robustness of the spectral method in theory and practice

Effective parameter retrieval for text models

Successful application to real-world text data

Abstract

This paper presents an algorithm for the unsupervised learning of latent variable models from unlabeled sets of data. We base our technique on spectral decomposition, providing a technique that proves to be robust both in theory and in practice. We also describe how to use this algorithm to learn the parameters of two well known text mining models: single topic model and Latent Dirichlet Allocation, providing in both cases an efficient technique to retrieve the parameters to feed the algorithm. We compare the results of our algorithm with those of existing algorithms on synthetic data, and we provide examples of applications to real world text corpora for both single topic model and LDA, obtaining meaningful results.