Regularization with the Smooth-Lasso procedure

TL;DR

This paper introduces the S-Lasso procedure for linear regression that promotes sparsity and accounts for correlations among predictors, demonstrating superior variable selection especially with highly correlated covariates, supported by theoretical and simulation results.

Contribution

The paper proposes the S-Lasso method, which combines sparsity and correlation awareness, with theoretical guarantees and improved variable selection over existing methods.

Findings

S-Lasso achieves variable selection consistency.

S-Lasso outperforms Lasso in correlated predictor scenarios.

The method provides an estimator for its effective degrees of freedom.

Abstract

We consider the linear regression problem. We propose the S-Lasso procedure to estimate the unknown regression parameters. This estimator enjoys sparsity of the representation while taking into account correlation between successive covariates (or predictors). The study covers the case when $p\gg n$, i.e. the number of covariates is much larger than the number of observations. In the theoretical point of view, for fixed $p$, we establish asymptotic normality and consistency in variable selection results for our procedure. When $p\geq n$, we provide variable selection consistency results and show that the S-Lasso achieved a Sparsity Inequality, i.e., a bound in term of the number of non-zero components of the oracle vector. It appears that the S-Lasso has nice variable selection properties compared to its challengers. Furthermore, we provide an estimator of the effective degree of freedom of the S-Lasso estimator. A simulation study shows that the S-Lasso performs better than the Lasso as far as variable selection is concerned especially when high correlations between successive covariates exist. This procedure also appears to be a good challenger to the Elastic-Net (Zou and Hastie, 2005).