Parallelism, Uniqueness, and Large-Sample Asymptotics for the Dantzig Selector

TL;DR

This paper establishes geometric conditions ensuring the uniqueness of the Dantzig selector in linear regression, analyzes its large-sample asymptotics, and compares these conditions with those for the lasso, providing new theoretical insights.

Contribution

It introduces a weak geometric condition for the uniqueness of the Dantzig selector, proves its almost sure validity under continuous predictor distributions, and derives its large-sample asymptotic behavior.

Findings

Uniqueness of the Dantzig selector is guaranteed under a weak geometric condition.

The asymptotic distribution of the Dantzig selector is generally non-normal.

The results hold for any number of predictors and observations, with asymptotics requiring fixed predictors.

Abstract

The Dantzig selector (Candes and Tao, 2007) is a popular l1-regularization method for variable selection and estimation in linear regression. We present a very weak geometric condition on the observed predictors which is related to parallelism and, when satisfied, ensures the uniqueness of Dantzig selector estimators. The condition holds with probability 1, if the predictors are drawn from a continuous distribution. We discuss the necessity of this condition for uniqueness and also provide a closely related condition which ensures uniqueness of lasso estimators (Tibshirani, 1996). Large sample asymptotics for the Dantzig selector, i.e. almost sure convergence and the asymptotic distribution, follow directly from our uniqueness results and a continuity argument. The limiting distribution of the Dantzig selector is generally non-normal. Though our asymptotic results require that the number of predictors is fixed (similar to (Knight and Fu, 2000)), our uniqueness results are valid for an arbitrary number of predictors and observations.