Tree Learning: Optimal Algorithms and Sample Complexity

TL;DR

This paper establishes optimal sample complexity bounds for learning hierarchical tree representations from labeled data, providing efficient algorithms for both PAC and online learning settings.

Contribution

It introduces tight bounds on sample complexity based on Natarajan and Littlestone dimensions and offers near-linear time algorithms for tree classifier construction.

Findings

Optimal sample complexity bounds for hierarchical tree learning.

Efficient near-linear time algorithms for tree classifier construction.

Application of Natarajan and Littlestone dimensions to bound complexity.

Abstract

We study the problem of learning a hierarchical tree representation of data from labeled samples, taken from an arbitrary (and possibly adversarial) distribution. Consider a collection of data tuples labeled according to their hierarchical structure. The smallest number of such tuples required in order to be able to accurately label subsequent tuples is of interest for data collection in machine learning. We present optimal sample complexity bounds for this problem in several learning settings, including (agnostic) PAC learning and online learning. Our results are based on tight bounds of the Natarajan and Littlestone dimensions of the associated problem. The corresponding tree classifiers can be constructed efficiently in near-linear time.