Modified Cross-Validation for Penalized High-Dimensional Linear Regression Models

TL;DR

This paper introduces a modified cross-validation method for penalized high-dimensional linear regression models, improving variable selection accuracy over traditional methods in various settings.

Contribution

It proposes a new cross-validation approach tailored for Lasso and Elastic Net models, enhancing penalty parameter selection in high-dimensional contexts.

Findings

Modified CV reduces noise variable inclusion

Performs well across diverse coefficient and correlation settings

Outperforms standard K-fold CV in simulations and real data

Abstract

In this paper, for Lasso penalized linear regression models in high-dimensional settings, we propose a modified cross-validation method for selecting the penalty parameter. The methodology is extended to other penalties, such as Elastic Net. We conduct extensive simulation studies and real data analysis to compare the performance of the modified cross-validation method with other methods. It is shown that the popular $K$-fold cross-validation method includes many noise variables in the selected model, while the modified cross-validation works well in a wide range of coefficient and correlation settings. Supplemental materials containing the computer code are available online.