The Proximity Operator of the Log-Sum Penalty

TL;DR

This paper derives an explicit expression for the proximity operator of the log-sum penalty, improving the accuracy and efficiency of algorithms in compressive sensing and low-rank optimization.

Contribution

It provides the first explicit formula for the proximity operator of the log-sum penalty, revealing limitations of the iteratively reweighted $ ext{l}_1$ method and characterizing its solutions.

Findings

Explicit form of the proximity operator for the log-sum penalty.

Discrepancies between the reweighted $ ext{l}_1$ method and the true proximity operator.

Characterization of the reweighted $ ext{l}_1$ solution based on initialization.

Abstract

The log-sum penalty is often adopted as a replacement for the $\ell_0$ pseudo-norm in compressive sensing and low-rank optimization. The hard-thresholding operator, i.e., the proximity operator of the $\ell_0$ penalty, plays an essential role in applications; similarly, we require an efficient method for evaluating the proximity operator of the log-sum penalty. Due to the nonconvexity of this function, its proximity operator is commonly computed through the iteratively reweighted $\ell_1$ method, which replaces the log-sum term with its first-order approximation. This paper reports that the proximity operator of the log-sum penalty actually has an explicit expression. With it, we show that the iteratively reweighted $\ell_1$ solution disagrees with the true proximity operator of the log-sum penalty in certain regions. As a by-product, the iteratively reweighted $\ell_1$ solution is precisely characterized in terms of the chosen initialization. We also give the explicit form of the proximity operator for the composition of the log-sum penalty with the singular value function, as seen in low-rank applications. These results should be useful in the development of efficient and accurate algorithms for optimization problems involving the log-sum penalty.