The Lasso, correlated design, and improved oracle inequalities

TL;DR

This paper introduces new oracle inequalities for the Lasso estimator in high-dimensional linear models by using entropy conditions, leading to improved prediction error bounds especially in correlated designs.

Contribution

It develops oracle inequalities based on entropy conditions, enhancing existing bounds and allowing for smaller tuning parameters and better handling of correlated designs.

Findings

Improved oracle inequalities for the Lasso estimator.

Enhanced prediction error bounds in correlated designs.

Demonstrates the benefit of entropy conditions over traditional methods.

Abstract

We study high-dimensional linear models and the $\ell_1$-penalized least squares estimator, also known as the Lasso estimator. In literature, oracle inequalities have been derived under restricted eigenvalue or compatibility conditions. In this paper, we complement this with entropy conditions which allow one to improve the dual norm bound, and demonstrate how this leads to new oracle inequalities. The new oracle inequalities show that a smaller choice for the tuning parameter and a trade-off between $\ell_1$-norms and small compatibility constants are possible. This implies, in particular for correlated design, improved bounds for the prediction error of the Lasso estimator as compared to the methods based on restricted eigenvalue or compatibility conditions only.