Adaptive elastic net and Separate Selection from Least Squares for ultra-high dimensional regression models

TL;DR

This paper analyzes the adaptive elastic net in ultra-high dimensional sparse linear regression, introduces SSLS to enhance prediction accuracy, and demonstrates its superior theoretical and empirical performance, especially in variable selection and bias reduction.

Contribution

It proposes the SSLS method that combines variable selection with OLS to improve prediction accuracy in ultra-high dimensional models, with proven theoretical properties.

Findings

Adaptive elastic net selection error probability decays exponentially.

SSLS bias decays exponentially, MSE approaches zero.

Empirical results show SSLS outperforms other methods in stock market index tracking.

Abstract

This paper studies the asymptotic properties of the adaptive elastic net in ultra-high dimensional sparse linear regression models and proposes a new method called SSLS (Separate Selection from Least Squares) to improve prediction accuracy. Besides, we prove that SSLS has the superior performance both in the theoretical part and empirical part. In this paper, we prove that the probability of adaptive elastic net selecting wrong variables can decays at an exponential rate with very few conditions. Irrepresentable Condition or similar constraint isn't necessary in our proof. We derive accurate bounds of bias and mean squared error (MSE) which both depend on the choice of parameters, and also show that there exists a bias of asymptotic normality of the adaptive elastic net. Furthermore, simulations and empirical part both show that the prediction accuracy of the penalized least squares requires more improvement. Therefore, we propose SSLS to improve the prediction. It selects variable first, reducing high dimension to low dimension by using the adaptive elastic net in this paper. In the second step, the coefficients are constructed based on the OLS estimation. We show that the bias of SSLS can decays at an exponential rate. Also, MSE decays to zero. Finally, we prove that the variable selection consistency of SSLS implies the asymptotic normality of SSLS. Simulations given in this paper illustrate the performance of the SSLS, adaptive elastic net and other penalized least squares. The index tracking problem in stock market is studied in the empirical part with other methods.