On tight bounds for the Lasso

TL;DR

This paper derives tight upper and lower bounds for the prediction error of the Lasso estimator under both random Gaussian and fixed design settings, linking errors to compatibility constants and betamin conditions.

Contribution

It provides the first comprehensive bounds for Lasso prediction error under mild conditions, including exact expressions in terms of compatibility constants.

Findings

Prediction error of Lasso is dominated by noiseless counterpart under Gaussian design.

Exact prediction error expressions are derived using compatibility constants.

Tight bounds are established for total variation penalized least squares estimator.

Abstract

We present upper and lower bounds for the prediction error of the Lasso. For the case of random Gaussian design, we show that under mild conditions the prediction error of the Lasso is up to smaller order terms dominated by the prediction error of its noiseless counterpart. We then provide exact expressions for the prediction error of the latter, in terms of compatibility constants. Here, we assume the active components of the underlying regression function satisfy some "betamin" condition. For the case of fixed design, we provide upper and lower bounds, again in terms of compatibility constants. As an example, we give an up to a logarithmic term tight bound for the least squares estimator with total variation penalty.