Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap

TL;DR

This paper analyzes the convergence rate of the Adaptive LASSO estimator to the oracle distribution in high-dimensional settings and demonstrates that bootstrap methods can significantly improve confidence interval coverage accuracy.

Contribution

It provides the first detailed analysis of the ALASSO convergence rate in high dimensions and shows bootstrap methods achieve second-order correctness for inference.

Findings

Convergence rate depends on penalty and initial estimator choices.

Oracle-based confidence intervals often have poor coverage.

Bootstrap confidence intervals significantly improve coverage accuracy.

Abstract

Zou [J. Amer. Statist. Assoc. 101 (2006) 1418-1429] proposed the Adaptive LASSO (ALASSO) method for simultaneous variable selection and estimation of the regression parameters, and established its oracle property. In this paper, we investigate the rate of convergence of the ALASSO estimator to the oracle distribution when the dimension of the regression parameters may grow to infinity with the sample size. It is shown that the rate critically depends on the choices of the penalty parameter and the initial estimator, among other factors, and that confidence intervals (CIs) based on the oracle limit law often have poor coverage accuracy. As an alternative, we consider the residual bootstrap method for the ALASSO estimators that has been recently shown to be consistent; cf. Chatterjee and Lahiri [J. Amer. Statist. Assoc. 106 (2011a) 608-625]. We show that the bootstrap applied to a suitable studentized version of the ALASSO estimator achieves second-order correctness, even when the dimension of the regression parameters is unbounded. Results from a moderately large simulation study show marked improvement in coverage accuracy for the bootstrap CIs over the oracle based CIs.