Lassoed Forests: Random Forests with Adaptive Lasso Post-selection
Jing Shang, James Bannon, Benjamin Haibe-Kains, Robert Tibshirani

TL;DR
This paper introduces a unified adaptive weighting framework for random forests and Lasso, demonstrating theoretical advantages and empirical improvements over traditional methods, especially in varying signal-to-noise conditions.
Contribution
It proposes a new adaptive weighting approach that combines random forests with Lasso selection, outperforming existing methods both theoretically and empirically.
Findings
Adaptive weighting can outperform standard and Lasso-weighted random forests.
Performance depends on the signal-to-noise ratio.
The method shows versatility across real-world datasets.
Abstract
Random forests are a statistical learning technique that use bootstrap aggregation to average high-variance and low-bias trees. Improvements to random forests, such as applying Lasso regression to the tree predictions, have been proposed in order to reduce model bias. However, these changes can sometimes degrade performance (e.g., an increase in mean squared error). In this paper, we show in theory that the relative performance of these two methods, standard and Lasso-weighted random forests, depends on the signal-to-noise ratio. We further propose a unified framework to combine random forests and Lasso selection by applying adaptive weighting and show mathematically that it can strictly outperform the other two methods. We compare the three methods through simulation, including bias-variance decomposition, error estimates evaluation, and variable importance analysis. We also show the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Methods and Inference · Financial Risk and Volatility Modeling · Data Stream Mining Techniques
