Guidelines for LASSO and derivatives use under different dependence and scale structures

TL;DR

This paper investigates how standardization affects LASSO performance in multivariate regression with various dependence and scale structures, providing guidelines for different scenarios.

Contribution

It offers a comprehensive analysis of standardization effects on LASSO under diverse dependence and scale conditions through simulations and real data comparisons.

Findings

Standardization impacts covariate selection accuracy.

Dependence structures influence LASSO's predictive performance.

Guidelines are proposed for choosing methods based on data characteristics.

Abstract

In a multivariate linear regression model with $p>1$ covariates, implementation of penalization techniques often implies a preliminary univariate standardization step. Although this prevents scale effects on the covariates selection procedure, possible dependence structures can be disrupted, leading to wrong results. This is particularly challenging in high-dimensional settings where $p \geq n$. In this paper, we analyze the standardization effect on the LASSO for different dependence-scales contexts by means of an extensive simulation study. Two distinct objectives are pursued: adequate covariate selection and proper predictive capability. Additionally, its behavior is compared with the one of some well-known or innovative competitors. This comparison is also extended to three real datasets facing different dependence-scales patterns. Eventually, we conclude with discussion and guidelines on the most suitable methodology for each case in terms of covariates selection or prediction.