A Stable Lasso
Mahdi Nouraie, Houying Zhu, Samuel Muller

TL;DR
This paper introduces a weighting scheme to enhance the stability of Lasso variable selection, especially with correlated predictors, demonstrating improved performance on simulated and real datasets.
Contribution
It proposes a simple, correlation-aware weighting technique to significantly improve Lasso's selection stability, extending to other regularization methods.
Findings
Improved stability of Lasso in correlated settings
Effective on both simulated and real datasets
Potential as a general-purpose regularization tool
Abstract
The Lasso has been widely used as a method for variable selection, valued for its simplicity and empirical performance. However, Lasso's selection stability deteriorates in the presence of correlated predictors. Several approaches have been developed to mitigate this limitation. In this paper, we provide a brief review of existing approaches, highlighting their limitations. We then propose a simple technique to improve the selection stability of Lasso by integrating a weighting scheme into the Lasso penalty function, where the weights are defined as an increasing function of a correlation-adjusted ranking that reflects the predictive power of predictors. Empirical evaluations on both simulated and real-world datasets demonstrate the efficacy of the proposed method. Additional numerical results demonstrate the effectiveness of the proposed approach in stabilizing other…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Methods and Inference · Advanced Bandit Algorithms Research · Bayesian Methods and Mixture Models
