On the Distribution of the Adaptive LASSO Estimator

TL;DR

This paper analyzes the distribution of the adaptive LASSO estimator in finite and large samples, revealing non-normality and slower convergence rates, which challenge the estimator's oracle property and include an impossibility result for distribution estimation.

Contribution

It provides the first detailed distributional analysis of the adaptive LASSO estimator, including finite-sample and asymptotic results, and questions the validity of its oracle property.

Findings

Distributions are highly non-normal regardless of tuning.

Convergence rate is slower than n^{-1/2} when performing consistent model selection.

An impossibility result is established for estimating the distribution function.

Abstract

We study the distribution of the adaptive LASSO estimator (Zou (2006)) in finite samples as well as in the large-sample limit. The large-sample distributions are derived both for the case where the adaptive LASSO estimator is tuned to perform conservative model selection as well as for the case where the tuning results in consistent model selection. We show that the finite-sample as well as the large-sample distributions are typically highly non-normal, regardless of the choice of the tuning parameter. The uniform convergence rate is also obtained, and is shown to be slower than $n^{-1/2}$ in case the estimator is tuned to perform consistent model selection. In particular, these results question the statistical relevance of the `oracle' property of the adaptive LASSO estimator established in Zou (2006). Moreover, we also provide an impossibility result regarding the estimation of the distribution function of the adaptive LASSO estimator.The theoretical results, which are obtained for a regression model with orthogonal design, are complemented by a Monte Carlo study using non-orthogonal regressors.