Uniform Asymptotics and Confidence Regions Based on the Adaptive Lasso with Partially Consistent Tuning

TL;DR

This paper analyzes the adaptive Lasso estimator with componentwise tuning in linear regression, providing uniform asymptotic confidence regions and exploring the effects of partial penalization and tuning on estimator properties.

Contribution

It offers a detailed asymptotic analysis of the adaptive Lasso with partial tuning, extending previous work to non-orthogonal regressors and explicit confidence set construction.

Findings

Explicit confidence sets with asymptotic coverage 1 and 0 depending on set inclusion.

Dependence of confidence set shape on regressor matrix and tuning deviations.

Generalization of prior orthogonal regressor results to broader settings.

Abstract

We consider the adaptive Lasso estimator with componentwise tuning in the framework of a low-dimensional linear regression model. In our setting, at least one of the components is penalized at the rate of consistent model selection and certain components may not be penalized at all. We perform a detailed study of the consistency properties and the asymptotic distribution which includes the effects of componentwise tuning within a so-called moving-parameter framework. These results enable us to explicitly provide a set $\mathcal{M}$ such that every open superset acts as a confidence set with uniform asymptotic coverage equal to 1, whereas removing an arbitrarily small open set along the boundary yields a confidence set with uniform asymptotic coverage equal to 0. The shape of the set $\mathcal{M}$ depends on the regressor matrix as well as the deviations within the componentwise tuning parameters. Our findings can be viewed as a broad generalization of P\"otscher & Schneider (2009, 2010) who considered distributional properties and confidence intervals based on components of the adaptive Lasso estimator for the case of orthogonal regressors.