Asymptotics of Proximity Operator for Squared Loss and Performance Prediction of Nonconvex Sparse Signal Recovery

TL;DR

This paper analyzes the asymptotic behavior of the proximity operator for squared loss in linear inverse problems, enabling accurate performance prediction of algorithms like Douglas-Rachford in compressed sensing, including nonconvex regularizations.

Contribution

It provides a novel asymptotic analysis of the proximity operator for squared loss and applies it to predict the performance of nonconvex sparse recovery algorithms.

Findings

Proximity operator output characterized by scalar random variable in large system limit

Accurate prediction of MSE performance for Douglas-Rachford algorithm in compressed sensing

Effective prediction for nonconvex regularizations like SCAD and MCP

Abstract

Proximal splitting-based convex optimization is a promising approach to linear inverse problems because we can use some prior knowledge of the unknown variables explicitly. An understanding of the behavior of the optimization algorithms would be important for the tuning of the parameters and the development of new algorithms. In this paper, we first analyze the asymptotic property of the proximity operator for the squared loss function, which appears in the update equations of some proximal splitting methods for linear inverse problems. Our analysis shows that the output of the proximity operator can be characterized with a scalar random variable in the large system limit. Moreover, we apply the asymptotic result to the prediction of optimization algorithms for compressed sensing. Simulation results demonstrate that the MSE performance of the Douglas-Rachford algorithm can be well predicted in compressed sensing with the $\ell_{1}$ optimization. We also examine the behavior of the prediction for the case with nonconvex smoothly clipped absolute deviation (SCAD) and minimax concave penalty (MCP) regularization.