TRUST: Transparent, Robust and Ultra-Sparse Trees

TL;DR

TRUST is a new regression tree model that combines high accuracy, interpretability, and transparency, using large language models for explanations, outperforming existing interpretable models and matching the accuracy of black-box methods.

Contribution

Introduces TRUST, a novel regression tree model that balances accuracy, interpretability, and transparency, leveraging large language models for user-friendly explanations.

Findings

TRUST outperforms CART, Lasso, and Node Harvest in predictive accuracy.

TRUST matches the accuracy of Random Forests.

TRUST offers significant improvements in interpretability and accuracy over M5'.

Abstract

Piecewise-constant regression trees remain popular for their interpretability, yet often lag behind black-box models like Random Forest in predictive accuracy. In this work, we introduce TRUST (Transparent, Robust, and Ultra-Sparse Trees), a novel regression tree model that combines the accuracy of Random Forests with the interpretability of shallow decision trees and sparse linear models. TRUST further enhances transparency by leveraging Large Language Models to generate tailored, user-friendly explanations. Extensive validation on synthetic and real-world benchmark datasets demonstrates that TRUST consistently outperforms other interpretable models -- including CART, Lasso, and Node Harvest -- in predictive accuracy, while matching the accuracy of Random Forest and offering substantial gains in both accuracy and interpretability over M5', a well-established model that is conceptually related.