Fast Algorithms for Learning Latent Variables in Graphical Models

TL;DR

This paper introduces fast, non-convex algorithms for estimating the low-rank component in Gaussian graphical models with latent variables, achieving optimal sample complexity and improved computational efficiency.

Contribution

The paper presents novel non-convex algorithms for low-rank estimation in latent variable Gaussian models, with theoretical guarantees and practical speed-ups.

Findings

Algorithms match the best possible sample complexity.

Achieve significant computational speed-ups.

Validated through numerical experiments.

Abstract

We study the problem of learning latent variables in Gaussian graphical models. Existing methods for this problem assume that the precision matrix of the observed variables is the superposition of a sparse and a low-rank component. In this paper, we focus on the estimation of the low-rank component, which encodes the effect of marginalization over the latent variables. We introduce fast, proper learning algorithms for this problem. In contrast with existing approaches, our algorithms are manifestly non-convex. We support their efficacy via a rigorous theoretical analysis, and show that our algorithms match the best possible in terms of sample complexity, while achieving computational speed-ups over existing methods. We complement our theory with several numerical experiments.