Lasso-Ridge Refitting: A Two-Stage Estimator for High-Dimensional Linear Regression

TL;DR

This paper introduces a novel Lasso--Ridge estimator that improves prediction accuracy and retains key advantages of Lasso in high-dimensional linear regression, validated through extensive simulations.

Contribution

The paper proposes a new two-stage Lasso--Ridge method that enhances prediction and estimation performance while maintaining Lasso's variable selection properties.

Findings

The estimator outperforms Lasso in prediction accuracy.

It retains prediction consistency and variable selection under mild conditions.

Simulation results confirm improved estimation and prediction performance.

Abstract

The least absolute shrinkage and selection operator (Lasso) is a popular method for high-dimensional statistics. However, it is known that the Lasso often has estimation bias and prediction error. To address such disadvantages, many alternatives and refitting strategies have been proposed and studied. This work introduces a novel Lasso--Ridge method. Our analysis indicates that the proposed estimator achieves improved prediction performance in a range of settings, including cases where the Lasso is tuned at its theoretical optimal rate \(\sqrt{\log(p)/n}\). Moreover, the proposed method retains several key advantages of the Lasso, such as prediction consistency and reliable variable selection under mild conditions. Through extensive simulations, we further demonstrate that our estimator outperforms the Lasso in both prediction and estimation accuracy, highlighting its potential as a powerful tool for high-dimensional linear regression.