Sparsity oracle inequalities for the Lasso

TL;DR

This paper establishes sparsity oracle inequalities for the Lasso estimator in high-dimensional, nonparametric regression, demonstrating its effectiveness even when the model dimension exceeds the sample size.

Contribution

It extends oracle inequalities for the Lasso to settings with random design, high dimensionality, and non-positive definite matrices, broadening its theoretical guarantees.

Findings

Oracle inequalities hold even when model dimension exceeds sample size.

Results apply to high-dimensional linear regression and nonparametric adaptive estimation.

The bounds are valid without requiring the regression matrix to be positive definite.

Abstract

This paper studies oracle properties of $\ell_1$-penalized least squares in nonparametric regression setting with random design. We show that the penalized least squares estimator satisfies sparsity oracle inequalities, i.e., bounds in terms of the number of non-zero components of the oracle vector. The results are valid even when the dimension of the model is (much) larger than the sample size and the regression matrix is not positive definite. They can be applied to high-dimensional linear regression, to nonparametric adaptive regression estimation and to the problem of aggregation of arbitrary estimators.