Towards optimal algorithms for the recovery of low-dimensional models with linear rates

TL;DR

This paper introduces a unifying framework for analyzing and optimizing algorithms that recover low-dimensional models from linear measurements, achieving linear convergence rates and applicable to sparse and neural network-based models.

Contribution

It proposes a class of generalized projected gradient descent algorithms with optimized Lipschitz constants for improved recovery performance.

Findings

Achieves linear convergence rates for low-dimensional recovery algorithms.

Provides a unified interpretation for sparse and neural network-based models.

Demonstrates effectiveness through experiments on synthetic and real data.

Abstract

We consider the problem of recovering elements of a low-dimensional model from linear measurements. From signal and image processing to inverse problems in data science, this question has been at the center of many applications. Lately, with the success of models and methods relying on deep neural networks, there has been a multiplication of different algorithms and recovery results. Comparing the performance of recovery algorithms becomes a complex task without a unifying framework. In this article, as a first step for the study of general algorithms for low-dimensional recovery, we study a class of generalized projected gradient descent algorithms that can recover a given low-dimensional model with linear rates. The obtained rates decouple the impact of the quality of the measurements with respect to the model from the geometry of the properties of the chosen generalized projection: we can directly measure performance through a restricted Lipschitz constant of the projection with respect to the low dimensional model. By optimizing this constant, we define an optimal generalized projected gradient descent. Our general approach provides an optimality result in the case of sparse recovery. Moreover, our framework allows for a common interpretation of linear rates of recovery in the context of both sparse models and models induced by some ``plug-and-play'' imaging methods that rely on deep neural networks. These rates of recovery are observed in experiments on synthetic and real data.