
TL;DR
This paper introduces a convex penalty method for tree ensembles, especially Random Forests, enabling variable selection and interpretability without sacrificing predictive accuracy, and requiring no additional tuning after model fitting.
Contribution
A novel Garrote-style convex penalty for tree ensembles that facilitates variable selection and interpretability while maintaining predictive performance without extra tuning.
Findings
Maintains predictive accuracy comparable to original Random Forests.
Enables visualization and interpretation of selected variables.
No additional tuning parameters needed after fitting.
Abstract
Variable selection for high-dimensional linear models has received a lot of attention lately, mostly in the context of l1-regularization. Part of the attraction is the variable selection effect: parsimonious models are obtained, which are very suitable for interpretation. In terms of predictive power, however, these regularized linear models are often slightly inferior to machine learning procedures like tree ensembles. Tree ensembles, on the other hand, lack usually a formal way of variable selection and are difficult to visualize. A Garrote-style convex penalty for trees ensembles, in particular Random Forests, is proposed. The penalty selects functional groups of nodes in the trees. These could be as simple as monotone functions of individual predictor variables. This yields a parsimonious function fit, which lends itself easily to visualization and interpretation. The predictive…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications
