A Rectification-Based Approach for Distilling Boosted Trees into Decision Trees

TL;DR

This paper introduces a rectification-based method for converting boosted trees into simpler decision trees, aiming to balance predictive accuracy with interpretability, and demonstrates its effectiveness through empirical evaluation.

Contribution

The paper proposes a novel rectification approach for distilling boosted trees into decision trees, offering an alternative to retraining-based methods.

Findings

The rectification approach yields competitive predictive performance.

Empirical results show improved interpretability with minimal loss in accuracy.

The method outperforms traditional retraining-based distillation techniques.

Abstract

We present a new approach for distilling boosted trees into decision trees, in the objective of generating an ML model offering an acceptable compromise in terms of predictive performance and interpretability. We explain how the correction approach called rectification can be used to implement such a distillation process. We show empirically that this approach provides interesting results, in comparison with an approach to distillation achieved by retraining the model.