Spectral Methods for Learning Multivariate Latent Tree Structure

TL;DR

This paper introduces a spectral recursive grouping algorithm for efficiently learning the structure of multivariate linear tree models from observed data, with guarantees that do not depend on the observed variables' dimensionality.

Contribution

The paper presents a novel spectral recursive grouping method with finite sample guarantees for structure learning in multivariate latent tree models.

Findings

Algorithm achieves exact recovery under certain conditions.

Sample complexity is independent of observed variable dimensionality.

Spectral quartet test effectively determines local topology.

Abstract

This work considers the problem of learning the structure of multivariate linear tree models, which include a variety of directed tree graphical models with continuous, discrete, and mixed latent variables such as linear-Gaussian models, hidden Markov models, Gaussian mixture models, and Markov evolutionary trees. The setting is one where we only have samples from certain observed variables in the tree, and our goal is to estimate the tree structure (i.e., the graph of how the underlying hidden variables are connected to each other and to the observed variables). We propose the Spectral Recursive Grouping algorithm, an efficient and simple bottom-up procedure for recovering the tree structure from independent samples of the observed variables. Our finite sample size bounds for exact recovery of the tree structure reveal certain natural dependencies on underlying statistical and structural properties of the underlying joint distribution. Furthermore, our sample complexity guarantees have no explicit dependence on the dimensionality of the observed variables, making the algorithm applicable to many high-dimensional settings. At the heart of our algorithm is a spectral quartet test for determining the relative topology of a quartet of variables from second-order statistics.