Under-determined linear systems and $\ell_q$-optimization thresholds

TL;DR

This paper investigates the theoretical limits of $ ext{ell}_q$-optimization for recovering sparse solutions in under-determined linear systems, extending the well-studied $ ext{ell}_1$ case to more general, non-convex relaxations.

Contribution

It provides a comprehensive analysis of the thresholds and limits of $ ext{ell}_q$-optimization for sparse recovery, offering insights beyond the convex $ ext{ell}_1$ approach.

Findings

Characterized the recovery thresholds for $ ext{ell}_q$-optimization.

Compared $ ext{ell}_q$ with $ ext{ell}_1$ in terms of sparsity recovery.

Explored the potential advantages of non-convex relaxations.

Abstract

Recent studies of under-determined linear systems of equations with sparse solutions showed a great practical and theoretical efficiency of a particular technique called $\ell_1$-optimization. Seminal works \cite{CRT,DOnoho06CS} rigorously confirmed it for the first time. Namely, \cite{CRT,DOnoho06CS} showed, in a statistical context, that $\ell_1$ technique can recover sparse solutions of under-determined systems even when the sparsity is linearly proportional to the dimension of the system. A followup \cite{DonohoPol} then precisely characterized such a linearity through a geometric approach and a series of work\cite{StojnicCSetam09,StojnicUpper10,StojnicEquiv10} reaffirmed statements of \cite{DonohoPol} through a purely probabilistic approach. A theoretically interesting alternative to $\ell_1$ is a more general version called $\ell_q$ (with an essentially arbitrary $q$). While $\ell_1$ is typically considered as a first available convex relaxation of sparsity norm $\ell_0$, $\ell_q,0\leq q\leq 1$, albeit non-convex, should technically be a tighter relaxation of $\ell_0$. Even though developing polynomial (or close to be polynomial) algorithms for non-convex problems is still in its initial phases one may wonder what would be the limits of an $\ell_q,0\leq q\leq 1$, relaxation even if at some point one can develop algorithms that could handle its non-convexity. A collection of answers to this and a few realted questions is precisely what we present in this paper.