Interpreting Models via Single Tree Approximation

TL;DR

This paper introduces a method to create a single decision tree that approximates complex models like random forests, aiding interpretability and reducing questionnaire length in medical settings.

Contribution

It presents a novel procedure for building stable, high-accuracy approximation trees for complex models, enhancing interpretability and practical application.

Findings

The method achieves high approximation accuracy.

It stabilizes tree structure through an improved splitting method.

Empirical results show effectiveness on real and simulated data.

Abstract

We propose a procedure to build a decision tree which approximates the performance of complex machine learning models. This single approximation tree can be used to interpret and simplify the predicting pattern of random forests (RFs) and other models. The use of a tree structure is particularly relevant in medical questionnaires where it enables an adaptive shortening of the questionnaire, reducing response burden. We study the asymptotic behavior of splits and introduce an improved splitting method designed to stabilize tree structure. Empirical studies on both simulation and real data sets illustrate that our method can simultaneously achieve high approximation power and stability.