Empowering Decision Trees via Shape Function Branching

TL;DR

This paper introduces Shape Generalized Trees (SGT), a new decision tree model that uses learnable shape functions at each node for richer, interpretable, non-linear feature partitioning, improving performance and interpretability.

Contribution

The paper proposes SGT, a novel decision tree generalization with learnable shape functions, along with ShapeCART, an efficient algorithm for training these trees, extending to multi-way and bivariate functions.

Findings

SGTs outperform traditional trees in accuracy and size.

SGTs are highly interpretable due to visualizable shape functions.

Proposed algorithms are efficient for training complex tree structures.

Abstract

Decision trees are prized for their interpretability and strong performance on tabular data. Yet, their reliance on simple axis-aligned linear splits often forces deep, complex structures to capture non-linear feature effects, undermining human comprehension of the constructed tree. To address this limitation, we propose a novel generalization of a decision tree, the Shape Generalized Tree (SGT), in which each internal node applies a learnable axis-aligned shape function to a single feature, enabling rich, non-linear partitioning in one split. As users can easily visualize each node's shape function, SGTs are inherently interpretable and provide intuitive, visual explanations of the model's decision mechanisms. To learn SGTs from data, we propose ShapeCART, an efficient induction algorithm for SGTs. We further extend the SGT framework to bivariate shape functions (S$^2$GT) and multi-way trees (SGT$_K$), and present Shape$^2$CART and ShapeCART$_K$, extensions to ShapeCART for learning S$^2$GTs and SGT$_K$s, respectively. Experiments on various datasets show that SGTs achieve superior performance with reduced model size compared to traditional axis-aligned linear trees.