Relaxed Sparse Eigenvalue Conditions for Sparse Estimation via Non-convex Regularized Regression

TL;DR

This paper demonstrates that non-convex regularizers, specifically sharp concave regularizers, enable sparse estimation under weaker conditions than traditional L1-regularization, with practical algorithms like coordinate descent effectively finding solutions.

Contribution

It introduces weaker sparse eigenvalue conditions for non-convex regularizers, expanding theoretical understanding and providing practical algorithms for sparse estimation.

Findings

Non-convex regularizers improve sparse estimation performance.

Weaker eigenvalue conditions suffice for global and approximate solutions.

Coordinate descent methods effectively find approximate solutions.

Abstract

Non-convex regularizers usually improve the performance of sparse estimation in practice. To prove this fact, we study the conditions of sparse estimations for the sharp concave regularizers which are a general family of non-convex regularizers including many existing regularizers. For the global solutions of the regularized regression, our sparse eigenvalue based conditions are weaker than that of L1-regularization for parameter estimation and sparseness estimation. For the approximate global and approximate stationary (AGAS) solutions, almost the same conditions are also enough. We show that the desired AGAS solutions can be obtained by coordinate descent (CD) based methods. Finally, we perform some experiments to show the performance of CD methods on giving AGAS solutions and the degree of weakness of the estimation conditions required by the sharp concave regularizers.