A critical review of LASSO and its derivatives for variable selection under dependence among covariates

TL;DR

This paper critically reviews LASSO's limitations for variable selection in dependent covariate settings, analyzing theoretical restrictions, practical drawbacks, and comparing derivatives and alternatives through extensive simulations.

Contribution

It provides a comprehensive analysis of LASSO's limitations under dependence, compares derivatives and alternatives, and offers practical guidance based on data characteristics.

Findings

LASSO has notable limitations with dependent covariates.

Certain derivatives and alternatives outperform LASSO in dependent scenarios.

Simulation results highlight the impact of dependence structures on variable selection methods.

Abstract

We study the limitations of the well known LASSO regression as a variable selector when there exists dependence structures among covariates. We analyze both the classic situation with $n\geq p$ and the high dimensional framework with $p>n$. Restrictive properties of this methodology to guarantee optimality, as well as the inconveniences in practice, are analyzed. Examples of these drawbacks are showed by means of a extensive simulation study, making use of different dependence scenarios. In order to search for improvements, a broad comparison with LASSO derivatives and alternatives is carried out. Eventually, we give some guidance about what procedures are the best in terms of the data nature.