Generalized Singular Value Thresholding

TL;DR

This paper introduces the Generalized Singular Value Thresholding (GSVT) operator, extending the classical SVT to nonconvex functions, enabling solutions to nonconvex low-rank minimization problems with broad applications.

Contribution

It proves GSVT can be obtained via the proximal operator of nonconvex functions on singular values and proposes a general solver for these functions, broadening low-rank minimization techniques.

Findings

GSVT generalizes SVT for nonconvex functions

A solver for the proximal operator of nonconvex functions is developed

GSVT enables solving nonconvex low-rank minimization problems

Abstract

This work studies the Generalized Singular Value Thresholding (GSVT) operator ${\text{Prox}}_{g}^{{\sigma}}(\cdot)$, \begin{equation*} {\text{Prox}}_{g}^{{\sigma}}(B)=\arg\min\limits_{X}\sum_{i=1}^{m}g(\sigma_{i}(X)) + \frac{1}{2}||X-B||_{F}^{2}, \end{equation*} associated with a nonconvex function $g$ defined on the singular values of $X$. We prove that GSVT can be obtained by performing the proximal operator of $g$ (denoted as $\text{Prox}_g(\cdot)$) on the singular values since $\text{Prox}_g(\cdot)$ is monotone when $g$ is lower bounded. If the nonconvex $g$ satisfies some conditions (many popular nonconvex surrogate functions, e.g., $\ell_p$-norm, $0<p<1$, of $\ell_0$-norm are special cases), a general solver to find $\text{Prox}_g(b)$ is proposed for any $b\geq0$. GSVT greatly generalizes the known Singular Value Thresholding (SVT) which is a basic subroutine in many convex low rank minimization methods. We are able to solve the nonconvex low rank minimization problem by using GSVT in place of SVT.