Prediction Suboptimality of the Lasso in Sparse Linear Regression

TL;DR

This paper investigates conditions under which the Lasso estimator performs suboptimally in high-dimensional sparse linear regression, revealing the influence of design matrix structure and stochastic factors on prediction accuracy.

Contribution

It identifies regimes where the Lasso's prediction performance can be improved and analyzes the role of Gaussian maxima and design matrix structure in this suboptimality.

Findings

Lasso can be suboptimal in certain tuning regimes.

Refinements can improve prediction performance.

Design matrix structure influences suboptimality.

Abstract

The choice of the tuning parameter in the Lasso is central to its statistical performance in high-dimensional linear regression. In this work, we study tuning regimes under which the Lasso exhibits suboptimal prediction performance, in the sense that a simple refinement improves upon it both on high-probability events and in mean squared prediction error. Our analysis shows that the relevant stochastic scale is governed by Gaussian maxima on the selected or localized support, which may be more informative than the universal rate in Lasso theory. We further illustrate how structural factors in the design matrix can influence the suboptimality phenomenon and discuss extensions to other estimators and more general noise structures.