Superconstant Inapproximability of Decision Tree Learning

TL;DR

This paper proves that properly PAC learning decision trees remains NP-hard even when the hypothesis tree is only required to be within a constant factor of the optimal size, establishing superconstant inapproximability.

Contribution

It demonstrates NP-hardness of near-optimal decision tree learning, introduces a new simpler proof of existing hardness, and develops a novel XOR lemma with sharp parameters.

Findings

NP-hardness persists within any constant factor of optimal decision trees

Introduces a new, simpler proof of decision tree learning hardness

Develops a new XOR lemma with sharp parameters for decision trees

Abstract

We consider the task of properly PAC learning decision trees with queries. Recent work of Koch, Strassle, and Tan showed that the strictest version of this task, where the hypothesis tree $T$ is required to be optimally small, is NP-hard. Their work leaves open the question of whether the task remains intractable if $T$ is only required to be close to optimal, say within a factor of 2, rather than exactly optimal. We answer this affirmatively and show that the task indeed remains NP-hard even if $T$ is allowed to be within any constant factor of optimal. More generally, our result allows for a smooth tradeoff between the hardness assumption and the inapproximability factor. As Koch et al.'s techniques do not appear to be amenable to such a strengthening, we first recover their result with a new and simpler proof, which we couple with a new XOR lemma for decision trees. While there is a large body of work on XOR lemmas for decision trees, our setting necessitates parameters that are extremely sharp, and are not known to be attainable by existing XOR lemmas. Our work also carries new implications for the related problem of Decision Tree Minimization.