New Error Analysis for Lasso

TL;DR

This paper introduces a new error bound for the Lasso estimator in high-dimensional linear regression, leveraging distributional assumptions to improve convergence rates and proposing a partitioning technique for sharper analysis.

Contribution

It provides a novel convergence rate bound for Lasso by exploiting distributional information and introduces a partitioning method for sharper analysis.

Findings

Improved convergence rate nearly $s/\sqrt{n}$ under certain conditions.

Applicable to various covariance matrices in high-dimensional data.

Enhances understanding of Lasso's error bounds with distributional assumptions.

Abstract

The Lasso is one of the most important approaches for parameter estimation and variable selection in high dimensional linear regression. At the heart of its success is the attractive rate of convergence result even when $p$, the dimension of the problem, is much larger than the sample size $n$. In particular, Bickel et al. (2009) showed that this rate, in terms of the $\ell_1$ norm, is of the order $s\sqrt{(\log p)/n}$ for a sparsity index $s$. In this paper, we obtain a new bound on the convergence rate by taking advantage of the distributional information of the model. Under the normality or sub-Gaussian assumption, the rate can be improved to nearly $s/\sqrt{n}$ for certain design matrices. We further outline a general partitioning technique that helps to derive sharper convergence rate for the Lasso. The result is applicable to many covariance matrices suitable for high-dimensional data analysis.