Properly learning decision trees in almost polynomial time

TL;DR

This paper presents a nearly polynomial time algorithm for properly and agnostically learning decision trees under the uniform distribution, improving upon previous runtimes and introducing new structural insights into decision trees.

Contribution

The authors develop an $n^{O(\log\log n)}$-time algorithm for learning decision trees, along with a new structural theorem that enhances understanding of decision tree influence and pruning.

Findings

Achieved a nearly polynomial time learning algorithm for decision trees.

Proved a new structural result that every decision tree can be pruned so all variables are influential.

Improved the runtime from $n^{O(\log n)}$ to $n^{O(\log\log n)}$ for the learning problem.

Abstract

We give an $n^{O(\log\log n)}$-time membership query algorithm for properly and agnostically learning decision trees under the uniform distribution over $\{\pm 1\}^n$. Even in the realizable setting, the previous fastest runtime was $n^{O(\log n)}$, a consequence of a classic algorithm of Ehrenfeucht and Haussler. Our algorithm shares similarities with practical heuristics for learning decision trees, which we augment with additional ideas to circumvent known lower bounds against these heuristics. To analyze our algorithm, we prove a new structural result for decision trees that strengthens a theorem of O'Donnell, Saks, Schramm, and Servedio. While the OSSS theorem says that every decision tree has an influential variable, we show how every decision tree can be "pruned" so that every variable in the resulting tree is influential.