A Gradient-Based Split Criterion for Highly Accurate and Transparent Model Trees

TL;DR

This paper introduces a new gradient-based split criterion for model trees that enhances predictive accuracy while maintaining transparency, combining the simplicity of shallow models with improved performance.

Contribution

The paper presents a novel split criterion for model trees that significantly improves predictive power without sacrificing transparency.

Findings

Achieves higher predictive accuracy than existing model trees

Maintains the transparency of simple models

Introduces mechanisms to enhance tree transparency

Abstract

Machine learning algorithms aim at minimizing the number of false decisions and increasing the accuracy of predictions. However, the high predictive power of advanced algorithms comes at the costs of transparency. State-of-the-art methods, such as neural networks and ensemble methods, often result in highly complex models that offer little transparency. We propose shallow model trees as a way to combine simple and highly transparent predictive models for higher predictive power without losing the transparency of the original models. We present a novel split criterion for model trees that allows for significantly higher predictive power than state-of-the-art model trees while maintaining the same level of simplicity. This novel approach finds split points which allow the underlying simple models to make better predictions on the corresponding data. In addition, we introduce multiple mechanisms to increase the transparency of the resulting trees.