Sharpness and well-conditioning of nonsmooth convex formulations in statistical signal recovery

TL;DR

This paper investigates the relationship between sample complexity and problem conditioning in convex formulations for various signal recovery tasks, showing that certain condition numbers become dimension-independent beyond a threshold, enabling efficient algorithms.

Contribution

It introduces a set of condition numbers related to sharpness in nonsmooth convex problems and demonstrates their dimension-independence in key signal recovery scenarios, leading to improved optimization methods.

Findings

Condition numbers become dimension-independent after sufficient samples.

A new restarted mirror descent method achieves nearly-dimension-independent linear convergence.

The results apply broadly to sharp convex functions in $\, ext{ell}_p$ or Schatten-$p$ norms.

Abstract

We study a sample complexity vs. conditioning tradeoff in modern signal recovery problems (including sparse recovery, low-rank matrix sensing, covariance estimation, and abstract phase retrieval), where convex optimization problems are built from sampled observations. We begin by introducing a set of condition numbers related to sharpness in $\ell_p$ or Schatten-$p$ norms ($p\in[1,2]$) of a nonsmooth formulation for these problems. Then, we show that these condition numbers become dimension independent constants in each of the example signal recovery problems once the sample size exceeds some constant multiple of the recovery threshold. Structurally, this result ensures that the inaccuracy in the recovered signal due to both observation noise and optimization error is well-controlled. Algorithmically, such a result ensures that a new restarted mirror descent method achieves nearly-dimension-independent linear convergence to the signal. This new first-order method is general and applies to any sharp convex function in an $\ell_p$ or Schatten-$p$ norm ($p\in[1,2]$).