Sensitivity of $\ell_{1}$ minimization to parameter choice

TL;DR

This paper investigates how sensitive $ ext{L}_1$ minimization (LASSO) is to parameter choices, revealing that small misestimations can cause large errors, especially in high-dimensional signal recovery.

Contribution

It provides a detailed analysis of the stability of $ ext{L}_1$ minimization with respect to its regularization parameter in the proximal denoising setting, highlighting potential instability issues.

Findings

Small underestimations of the LASSO parameter can significantly increase error.

A 50% underestimate can cause the error to grow by a factor of 10^9.

The analysis is supported by numerical simulations demonstrating instability scenarios.

Abstract

The use of generalized LASSO is a common technique for recovery of structured high-dimensional signals. Each generalized LASSO program has a governing parameter whose optimal value depends on properties of the data. At this optimal value, compressed sensing theory explains why LASSO programs recover structured high-dimensional signals with minimax order-optimal error. Unfortunately in practice, the optimal choice is generally unknown and must be estimated. Thus, we investigate stability of each LASSO program with respect to its governing parameter. Our goal is to aid the practitioner in answering the following question: given real data, which LASSO program should be used? We take a step towards answering this by analyzing the case where the measurement matrix is identity (the so-called proximal denoising setup) and we use $\ell_{1}$ regularization. For each LASSO program, we specify settings in which that program is provably unstable with respect to its governing parameter. We support our analysis with detailed numerical simulations. For example, there are settings where a 0.1% underestimate of a LASSO parameter can increase the error significantly; and a 50% underestimate can cause the error to increase by a factor of $10^{9}$.