Optimality of 1-norm regularization among weighted 1-norms for sparse recovery: a case study on how to find optimal regularizations

TL;DR

This paper investigates the optimality of 1-norm and weighted 1-norm regularizations for sparse recovery, establishing their optimality within certain frameworks and highlighting challenges in higher dimensions.

Contribution

It introduces a framework to identify the best convex regularizer for low-dimensional models and proves the 1-norm's optimality among weighted 1-norms for sparse recovery.

Findings

1-norm is optimal for sparse recovery in 3D under certain notions.

The 1-norm is optimal among weighted 1-norms for sparse recovery.

Generalization to higher dimensions remains challenging.

Abstract

The 1-norm was proven to be a good convex regularizer for the recovery of sparse vectors from under-determined linear measurements. It has been shown that with an appropriate measurement operator, a number of measurements of the order of the sparsity of the signal (up to log factors) is sufficient for stable and robust recovery. More recently, it has been shown that such recovery results can be generalized to more general low-dimensional model sets and (convex) regularizers. These results lead to the following question: to recover a given low-dimensional model set from linear measurements, what is the "best" convex regularizer? To approach this problem, we propose a general framework to define several notions of "best regularizer" with respect to a low-dimensional model. We show in the minimal case of sparse recovery in dimension 3 that the 1-norm is optimal for these notions. However, generalization of such results to the n-dimensional case seems out of reach. To tackle this problem, we propose looser notions of best regularizer and show that the 1-norm is optimal among weighted 1-norms for sparse recovery within this framework.