Robustifying Algorithms of Learning Latent Trees with Vector Variables

TL;DR

This paper advances the learning of Gaussian latent tree structures with vector data by reducing sample complexity, introducing robustness against corruptions, and establishing fundamental impossibility limits.

Contribution

It generalizes existing algorithms to unbounded effective depth, robustifies multiple algorithms against corruptions, and provides the first instance-dependent impossibility result for latent tree learning.

Findings

Chow-Liu initialization reduces sample complexity from exponential to logarithmic in tree diameter.

Robust algorithms tolerate corruptions up to the square root of the number of clean samples.

First instance-dependent impossibility result for structure learning of latent trees.

Abstract

We consider learning the structures of Gaussian latent tree models with vector observations when a subset of them are arbitrarily corrupted. First, we present the sample complexities of Recursive Grouping (RG) and Chow-Liu Recursive Grouping (CLRG) without the assumption that the effective depth is bounded in the number of observed nodes, significantly generalizing the results in Choi et al. (2011). We show that Chow-Liu initialization in CLRG greatly reduces the sample complexity of RG from being exponential in the diameter of the tree to only logarithmic in the diameter for the hidden Markov model (HMM). Second, we robustify RG, CLRG, Neighbor Joining (NJ) and Spectral NJ (SNJ) by using the truncated inner product. These robustified algorithms can tolerate a number of corruptions up to the square root of the number of clean samples. Finally, we derive the first known instance-dependent impossibility result for structure learning of latent trees. The optimalities of the robust version of CLRG and NJ are verified by comparing their sample complexities and the impossibility result.