Inherently Interpretable Tree Ensemble Learning

TL;DR

This paper introduces inherently interpretable tree ensemble models using shallow trees, providing a new interpretation algorithm and strategies to improve transparency without sacrificing predictive accuracy.

Contribution

It presents a novel approach to make ensemble models inherently interpretable by using shallow trees and developing an interpretation algorithm, enhancing transparency and sometimes improving performance.

Findings

The proposed methods achieve a better balance between interpretability and accuracy.

Shallow tree ensembles can be represented as generalized additive models.

Experiments demonstrate improved interpretability with competitive predictive performance.

Abstract

Tree ensemble models like random forests and gradient boosting machines are widely used in machine learning due to their excellent predictive performance. However, a high-performance ensemble consisting of a large number of decision trees lacks sufficient transparency and explainability. In this paper, we demonstrate that when shallow decision trees are used as base learners, the ensemble learning algorithms can not only become inherently interpretable subject to an equivalent representation as the generalized additive models but also sometimes lead to better generalization performance. First, an interpretation algorithm is developed that converts the tree ensemble into the functional ANOVA representation with inherent interpretability. Second, two strategies are proposed to further enhance the model interpretability, i.e., by adding constraints in the model training stage and post-hoc effect pruning. Experiments on simulations and real-world datasets show that our proposed methods offer a better trade-off between model interpretation and predictive performance, compared with its counterpart benchmarks.