Optimal Rates for Learning Hidden Tree Structures

TL;DR

This paper establishes the fundamental limits and guarantees for learning hidden tree-structured graphical models from noisy data, showing that the sample complexity depends on a key information threshold and that the Chow-Liu algorithm is optimal under these conditions.

Contribution

It introduces the information threshold as a critical quantity for sample complexity and proves the optimality of the Chow-Liu algorithm in noisy settings, including non-i.i.d. noise.

Findings

Sample complexity scales inversely with the square of the information threshold.

Chow-Liu algorithm achieves optimal rates with respect to the information threshold.

No algorithm can recover the structure with probability > 1/2 below a certain sample size.

Abstract

We provide high probability finite sample complexity guarantees for hidden non-parametric structure learning of tree-shaped graphical models, whose hidden and observable nodes are discrete random variables with either finite or countable alphabets. We study a fundamental quantity called the (noisy) information threshold, which arises naturally from the error analysis of the Chow-Liu algorithm and, as we discuss, provides explicit necessary and sufficient conditions on sample complexity, by effectively summarizing the difficulty of the tree-structure learning problem. Specifically, we show that the finite sample complexity of the Chow-Liu algorithm for ensuring exact structure recovery from noisy data is inversely proportional to the information threshold squared (provided it is positive), and scales almost logarithmically relative to the number of nodes over a given probability of failure. Conversely, we show that, if the number of samples is less than an absolute constant times the inverse of information threshold squared, then no algorithm can recover the hidden tree structure with probability greater than one half. As a consequence, our upper and lower bounds match with respect to the information threshold, indicating that it is a fundamental quantity for the problem of learning hidden tree-structured models. Further, the Chow-Liu algorithm with noisy data as input achieves the optimal rate with respect to the information threshold. Lastly, as a byproduct of our analysis, we resolve the problem of tree structure learning in the presence of non-identically distributed observation noise, providing conditions for convergence of the Chow-Liu algorithm under this setting, as well.