Penalized Split Criteria for Interpretable Trees

TL;DR

This paper introduces penalized split criteria for classification and regression trees to enhance interpretability by limiting feature subset expansion, achieving more understandable models with minimal loss increase.

Contribution

It proposes new penalization techniques for tree splits that improve interpretability without significantly sacrificing predictive performance.

Findings

More interpretable trees than CART

Minimal increase in out-of-sample loss

Effective on real datasets

Abstract

This paper describes techniques for growing classification and regression trees designed to induce visually interpretable trees. This is achieved by penalizing splits that extend the subset of features used in a particular branch of the tree. After a brief motivation, we summarize existing methods and introduce new ones, providing illustrative examples throughout. Using a number of real classification and regression datasets, we find that these procedures can offer more interpretable fits than the CART methodology with very modest increases in out-of-sample loss.