Hierarchical Methods of Moments

TL;DR

This paper introduces a hierarchical approach to spectral methods of moments, replacing tensor decomposition with joint diagonalization, improving robustness and efficiency in latent variable model learning.

Contribution

The paper presents a novel hierarchical method that enhances robustness and speed of spectral moments techniques through approximate joint diagonalization.

Findings

Outperforms previous tensor methods in speed

Achieves better model quality in topic modeling

Demonstrates robustness to model misspecification

Abstract

Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models. Despite their theoretical appeal, the applicability of these methods to real data is still limited due to a lack of robustness to model misspecification. In this paper we present a hierarchical approach to methods of moments to circumvent such limitations. Our method is based on replacing the tensor decomposition step used in previous algorithms with approximate joint diagonalization. Experiments on topic modeling show that our method outperforms previous tensor decomposition methods in terms of speed and model quality.