A Stable Lasso

TL;DR

This paper introduces a weighting scheme to enhance the stability of Lasso variable selection, especially with correlated predictors, demonstrating improved performance on simulated and real datasets.

Contribution

It proposes a simple, correlation-aware weighting technique to significantly improve Lasso's selection stability, extending to other regularization methods.

Findings

Improved stability of Lasso in correlated settings

Effective on both simulated and real datasets

Potential as a general-purpose regularization tool

Abstract

The Lasso has been widely used as a method for variable selection, valued for its simplicity and empirical performance. However, Lasso's selection stability deteriorates in the presence of correlated predictors. Several approaches have been developed to mitigate this limitation. In this paper, we provide a brief review of existing approaches, highlighting their limitations. We then propose a simple technique to improve the selection stability of Lasso by integrating a weighting scheme into the Lasso penalty function, where the weights are defined as an increasing function of a correlation-adjusted ranking that reflects the predictive power of predictors. Empirical evaluations on both simulated and real-world datasets demonstrate the efficacy of the proposed method. Additional numerical results demonstrate the effectiveness of the proposed approach in stabilizing other regularization-based selection methods, indicating its potential as a general-purpose solution.