LHT: Statistically-Driven Oblique Decision Trees for Interpretable Classification

TL;DR

LHT introduces a non-iterative, statistically-driven oblique decision tree model that constructs hyperplanes directly from feature expectations, offering interpretability and competitive accuracy for classification tasks.

Contribution

The paper presents a novel, deterministic method for constructing oblique decision trees using statistical feature differences, eliminating iterative optimization.

Findings

LHT achieves competitive accuracy on benchmark datasets.

The method has a time complexity of O(mnd), making it computationally feasible.

Explicit feature weighting enhances interpretability of the model.

Abstract

We introduce the Learning Hyperplane Tree (LHT), a novel oblique decision tree model designed for expressive and interpretable classification. LHT fundamentally distinguishes itself through a non-iterative, statistically-driven approach to constructing splitting hyperplanes. Unlike methods that rely on iterative optimization or heuristics, LHT directly computes the hyperplane parameters, which are derived from feature weights based on the differences in feature expectations between classes within each node. This deterministic mechanism enables a direct and well-defined hyperplane construction process. Predictions leverage a unique piecewise linear membership function within leaf nodes, obtained via local least-squares fitting. We formally analyze the convergence of the LHT splitting process, ensuring that each split yields meaningful, non-empty partitions. Furthermore, we establish that the time complexity for building an LHT up to depth $d$ is $O(mnd)$, demonstrating the practical feasibility of constructing trees with powerful oblique splits using this methodology. The explicit feature weighting at each split provides inherent interpretability. Experimental results on benchmark datasets demonstrate LHT's competitive accuracy, positioning it as a practical, theoretically grounded, and interpretable alternative in the landscape of tree-based models. The implementation of the proposed method is available at https://github.com/Hongyi-Li-sz/LHT_model.