On the Conditions of Sparse Parameter Estimation via Log-Sum Penalty Regularization

TL;DR

This paper provides theoretical conditions under which Log-Sum Penalty (LSP) regularization effectively estimates sparse parameters in high-dimensional settings, supported by an efficient algorithm with consistent results.

Contribution

It offers the first theoretical analysis of LSP regularization's sampling requirements and introduces an efficient algorithm with improved consistency over $ ext{l}_1$-regularization.

Findings

O(s) sampling size suffices for proper LSP

Proposed algorithm yields consistent estimates

LSP requires less restrictive conditions than $ ext{l}_1$-regularization

Abstract

For high-dimensional sparse parameter estimation problems, Log-Sum Penalty (LSP) regularization effectively reduces the sampling sizes in practice. However, it still lacks theoretical analysis to support the experience from previous empirical study. The analysis of this article shows that, like $\ell_0$-regularization, $O(s)$ sampling size is enough for proper LSP, where $s$ is the non-zero components of the true parameter. We also propose an efficient algorithm to solve LSP regularization problem. The solutions given by the proposed algorithm give consistent parameter estimations under less restrictive conditions than $\ell_1$-regularization.