A theory of optimal convex regularization for low-dimensional recovery

TL;DR

This paper develops a theoretical framework to identify the most effective convex regularizers for recovering low-dimensional structures from limited measurements, demonstrating the optimality of well-known norms like ℓ1 and nuclear norm.

Contribution

It introduces a formal definition of optimal convex regularizers based on compliance measures and provides analytical expressions to identify the best regularizers for various low-dimensional models.

Findings

ℓ1-norm is optimal for sparse recovery under certain measures

Nuclear norm is optimal for low-rank matrix recovery

Analytical expressions enable the construction of optimal regularizers for complex models

Abstract

We consider the problem of recovering elements of a low-dimensional model from under-determined linear measurements. To perform recovery, we consider the minimization of a convex regularizer subject to a data fit constraint. Given a model, we ask ourselves what is the "best" convex regularizer to perform its recovery. To answer this question, we define an optimal regularizer as a function that maximizes a compliance measure with respect to the model. We introduce and study several notions of compliance. We give analytical expressions for compliance measures based on the best-known recovery guarantees with the restricted isometry property. These expressions permit to show the optimality of the ${\ell}$1-norm for sparse recovery and of the nuclear norm for low-rank matrix recovery for these compliance measures. We also investigate the construction of an optimal convex regularizer using the examples of sparsity in levels and of sparse plus low-rank models.