Fast Linear Model Trees by PILOT
Jakob Raymaekers, Peter J. Rousseeuw, Tim Verdonck, Ruicong Yao

TL;DR
PILOT is a fast, regularized, and interpretable algorithm for linear model trees that improves scalability and performance over existing methods, with proven consistency and better results on various datasets.
Contribution
Introduces PILOT, a novel linear model tree algorithm that is fast, regularized, stable, and scalable, with theoretical consistency guarantees.
Findings
PILOT outperforms standard decision trees and other linear model trees on multiple datasets.
PILOT has the same low complexity as CART but without pruning.
Proven consistency in additive models with polynomial convergence rate when data is linear.
Abstract
Linear model trees are regression trees that incorporate linear models in the leaf nodes. This preserves the intuitive interpretation of decision trees and at the same time enables them to better capture linear relationships, which is hard for standard decision trees. But most existing methods for fitting linear model trees are time consuming and therefore not scalable to large data sets. In addition, they are more prone to overfitting and extrapolation issues than standard regression trees. In this paper we introduce PILOT, a new algorithm for linear model trees that is fast, regularized, stable and interpretable. PILOT trains in a greedy fashion like classic regression trees, but incorporates an boosting approach and a model selection rule for fitting linear models in the nodes. The abbreviation PILOT stands for ecewise inear rganic ree, where `organic' refers to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBayesian Modeling and Causal Inference · Machine Learning and Data Classification · Statistical Methods and Inference
MethodsPruning
