CART-ELC: Oblique Decision Tree Induction via Exhaustive Search

TL;DR

CART-ELC is a new algorithm for inducing oblique decision trees using exhaustive search on a limited set of hyperplanes, achieving competitive accuracy and simpler, more interpretable trees on small datasets.

Contribution

The paper introduces CART-ELC, a novel exhaustive search algorithm for oblique decision trees that balances computational complexity with improved classification performance.

Findings

CART-ELC achieves statistically significant accuracy improvements.

It produces shallower, more interpretable trees.

Performance is competitive on small datasets.

Abstract

Oblique decision trees have attracted attention due to their potential for improved classification performance over traditional axis-aligned decision trees. However, methods that rely on exhaustive search to find oblique splits face computational challenges. As a result, they have not been widely explored. We introduce a novel algorithm, Classification and Regression Tree - Exhaustive Linear Combinations (CART-ELC), for inducing oblique decision trees that performs an exhaustive search on a restricted set of hyperplanes. We then investigate the algorithm's computational complexity and its predictive capabilities. Our results demonstrate that CART-ELC consistently achieves competitive performance on small datasets, often yielding statistically significant improvements in classification accuracy relative to existing decision tree induction algorithms, while frequently producing shallower, simpler, and thus more interpretable trees.