Interpreting Tree Ensembles with inTrees

TL;DR

The paper introduces the inTrees framework that extracts and simplifies rules from tree ensembles like random forests and boosted trees, enhancing interpretability for classification and regression tasks.

Contribution

It provides a comprehensive method for extracting, pruning, and selecting rules from various tree ensembles, and introduces the simplified tree ensemble learner (STEL) for prediction.

Findings

Effective rule extraction from different tree ensembles

Applicable to classification and regression problems

Implemented in the inTrees R package

Abstract

Tree ensembles such as random forests and boosted trees are accurate but difficult to understand, debug and deploy. In this work, we provide the inTrees (interpretable trees) framework that extracts, measures, prunes and selects rules from a tree ensemble, and calculates frequent variable interactions. An rule-based learner, referred to as the simplified tree ensemble learner (STEL), can also be formed and used for future prediction. The inTrees framework can applied to both classification and regression problems, and is applicable to many types of tree ensembles, e.g., random forests, regularized random forests, and boosted trees. We implemented the inTrees algorithms in the "inTrees" R package.