A Unifying Analysis of Projected Gradient Descent for $\ell_p$-constrained Least Squares

TL;DR

This paper provides a comprehensive analysis of Projected Gradient Descent for p-constrained least squares problems in compressed sensing, offering convergence guarantees across all p in [0,1] and comparing robustness and accuracy.

Contribution

It unifies and generalizes convergence analysis for PGD in p-constrained problems, extending results for iterative hard and soft thresholding algorithms.

Findings

Convergence guarantees hold for all 0  p  1 under RIP.

Accuracy conditions become stricter as p increases.

Robustness to noise decreases with larger p.

Abstract

In this paper we study the performance of the Projected Gradient Descent(PGD) algorithm for $\ell_{p}$-constrained least squares problems that arise in the framework of Compressed Sensing. Relying on the Restricted Isometry Property, we provide convergence guarantees for this algorithm for the entire range of $0\leq p\leq1$, that include and generalize the existing results for the Iterative Hard Thresholding algorithm and provide a new accuracy guarantee for the Iterative Soft Thresholding algorithm as special cases. Our results suggest that in this group of algorithms, as $p$ increases from zero to one, conditions required to guarantee accuracy become stricter and robustness to noise deteriorates.