Sparse recovery via nonconvex regularized $M$-estimators over $\ell_q$-balls

TL;DR

This paper analyzes the recovery properties of nonconvex regularized M-estimators for sparse signals, providing theoretical bounds, an efficient algorithm with linear convergence, and demonstrating superior performance through numerical experiments.

Contribution

It offers new theoretical recovery bounds for nonconvex regularized M-estimators and introduces an efficient proximal gradient algorithm with proven convergence rates.

Findings

Recovery bounds for stationary points under restricted strong convexity

Proximal gradient method achieves linear convergence

Numerical experiments confirm theoretical predictions and advantages of the approach

Abstract

In this paper, we analyse the recovery properties of nonconvex regularized $M$-estimators, under the assumption that the true parameter is of soft sparsity. In the statistical aspect, we establish the recovery bound for any stationary point of the nonconvex regularized $M$-estimator, under restricted strong convexity and some regularity conditions on the loss function and the regularizer, respectively. In the algorithmic aspect, we slightly decompose the objective function and then solve the nonconvex optimization problem via the proximal gradient method, which is proved to achieve a linear convergence rate. In particular, we note that for commonly-used regularizers such as SCAD and MCP, a simpler decomposition is applicable thanks to our assumption on the regularizer, which helps to construct the estimator with better recovery performance. Finally, we demonstrate our theoretical consequences and the advantage of the assumption by several numerical experiments on the corrupted errors-in-variables linear regression model. Simulation results show remarkable consistency with our theory under high-dimensional scaling.