Sharp minimax optimality of LASSO and SLOPE under double sparsity assumption

TL;DR

This paper rigorously establishes the sharp minimax optimality of LASSO and SLOPE under double sparsity without RIP, relying on sparse group normalization and RE-type conditions, with high probability results for various random matrices.

Contribution

It introduces novel sparse group RE conditions and demonstrates their sufficiency for optimality, extending results to random design settings with broad distribution assumptions.

Findings

Optimality of LASSO and SLOPE under double sparsity is proven.

New sparse group RE conditions are introduced and analyzed.

High probability bounds are established for random matrices.

Abstract

This paper introduces a rigorous approach to establish the sharp minimax optimalities of both LASSO and SLOPE within the framework of double sparse structures, notably without relying on RIP-type conditions. Crucially, our findings illuminate that the achievement of these optimalities is fundamentally anchored in a sparse group normalization condition, complemented by several novel sparse group restricted eigenvalue (RE)-type conditions introduced in this study. We further provide a comprehensive comparative analysis of these eigenvalue conditions. Furthermore, we demonstrate that these conditions hold with high probability across a wide range of random matrices. Our exploration extends to encompass the random design, where we prove the random design properties and optimal sample complexity under both weak moment distribution and sub-Gaussian distribution.