A Unified Approach to Extract Interpretable Rules from Tree Ensembles via Integer Programming

TL;DR

This paper presents a novel integer programming-based method to extract concise, interpretable rule sets from complex tree ensemble models, maintaining high predictive accuracy for classification and regression tasks on tabular and time series data.

Contribution

It introduces a flexible, optimization-based approach for rule extraction that improves interpretability without sacrificing predictive performance.

Findings

Competitive predictive performance compared to other rule extraction methods.

Effective extraction of interpretable rules for time series data.

Statistically significant improvements in model interpretability.

Abstract

Tree ensembles are very popular machine learning models, known for their effectiveness in supervised classification and regression tasks. Their performance derives from aggregating predictions of multiple decision trees, which are renowned for their interpretability properties. However, tree ensemble models do not reliably exhibit interpretable output. Our work aims to extract an optimized list of rules from a trained tree ensemble, providing the user with a condensed, interpretable model that retains most of the predictive power of the full model. Our approach consists of solving a set partitioning problem formulated through Integer Programming. The proposed method works with either tabular or time series data, for both classification and regression tasks, and its flexible formulation can include any arbitrary loss or regularization functions. Our extensive computational experiments offer statistically significant evidence that our method is competitive with other rule extraction methods in terms of predictive performance and fidelity towards the tree ensemble. Moreover, we empirically show that the proposed method effectively extracts interpretable rules from tree ensemble that are designed for time series data.