Extremely Fast Decision Tree

TL;DR

The paper introduces Hoeffding Anytime Tree, an incremental decision tree algorithm that is more statistically efficient and achieves higher accuracy than existing methods like Hoeffding Tree, especially on large datasets, with minimal additional computational cost.

Contribution

It presents Hoeffding Anytime Tree, a novel incremental decision tree algorithm that outperforms Hoeffding Tree in accuracy and efficiency, and demonstrates its practical implementation and benefits.

Findings

Significantly higher prequential accuracy on large datasets

Produces the asymptotic batch tree in the limit

Resilient to concept drift

Abstract

We introduce a novel incremental decision tree learning algorithm, Hoeffding Anytime Tree, that is statistically more efficient than the current state-of-the-art, Hoeffding Tree. We demonstrate that an implementation of Hoeffding Anytime Tree---"Extremely Fast Decision Tree", a minor modification to the MOA implementation of Hoeffding Tree---obtains significantly superior prequential accuracy on most of the largest classification datasets from the UCI repository. Hoeffding Anytime Tree produces the asymptotic batch tree in the limit, is naturally resilient to concept drift, and can be used as a higher accuracy replacement for Hoeffding Tree in most scenarios, at a small additional computational cost.