A Faster Drop-in Implementation for Leaf-wise Exact Greedy Induction of Decision Tree Using Pre-sorted Deque

TL;DR

This paper introduces a new implementation for decision trees that uses pre-sorted deques to significantly speed up leaf-wise greedy induction, maintaining accuracy while reducing computational complexity.

Contribution

It presents a novel, efficient implementation using pre-sorted deques that eliminates re-sorting at each node, enabling faster decision tree induction without sacrificing accuracy.

Findings

Achieves O(kn) time complexity for split finding

Requires only O(1) extra memory per node

Maintains identical classification results as standard methods

Abstract

This short article presents a new implementation for decision trees. By introducing pre-sorted deques, the leaf-wise greedy tree growing strategy no longer needs to re-sort data at each node, and takes O(kn) time and O(1) extra memory locating the best split and branching. The consistent, superior performance - plus its simplicity and guarantee in producing the same classification results as the standard decision trees - makes the new implementation a drop-in replacement for depth-wise tree induction with strong performance.