On the conditions used to prove oracle results for the Lasso

TL;DR

This paper clarifies the relationships among various conditions used to establish oracle inequalities for the Lasso, showing that the restricted eigenvalue and compatibility conditions are sufficient for broad classes of design matrices, extending the applicability of Lasso's optimality.

Contribution

It unifies and compares different assumptions for the Lasso, demonstrating that the restricted eigenvalue and compatibility conditions are sufficient for oracle results in more general settings.

Findings

Restricted eigenvalue and compatibility conditions suffice for oracle results.

These conditions apply to a broad class of design matrices.

Optimality of Lasso extends beyond coherence and restricted isometry assumptions.

Abstract

Oracle inequalities and variable selection properties for the Lasso in linear models have been established under a variety of different assumptions on the design matrix. We show in this paper how the different conditions and concepts relate to each other. The restricted eigenvalue condition (Bickel et al., 2009) or the slightly weaker compatibility condition (van de Geer, 2007) are sufficient for oracle results. We argue that both these conditions allow for a fairly general class of design matrices. Hence, optimality of the Lasso for prediction and estimation holds for more general situations than what it appears from coherence (Bunea et al, 2007b,c) or restricted isometry (Candes and Tao, 2005) assumptions.