A Unified Convex Surrogate for the Schatten-$p$ Norm

TL;DR

This paper introduces a unified convex surrogate for the Schatten-$p$ norm that enables scalable, convex, and smooth optimization for low-rank matrix problems across all $p>0$, improving over existing methods.

Contribution

It establishes a novel equivalence between Schatten-$p$ norms and factor matrix norms, extending to multiple factors, and develops an efficient convex optimization algorithm.

Findings

The proposed method outperforms state-of-the-art algorithms in matrix completion tasks.

The approach is scalable to large datasets due to convexity and smoothness of factor norms.

Experiments demonstrate superior accuracy and competitive speed.

Abstract

The Schatten-$p$ norm ($0<p<1$) has been widely used to replace the nuclear norm for better approximating the rank function. However, existing methods are either 1) not scalable for large scale problems due to relying on singular value decomposition (SVD) in every iteration, or 2) specific to some $p$ values, e.g., $1/2$, and $2/3$. In this paper, we show that for any $p$, $p_1$, and $p_2 >0$ satisfying $1/p=1/p_1+1/p_2$, there is an equivalence between the Schatten-$p$ norm of one matrix and the Schatten-$p_1$ and the Schatten-$p_2$ norms of its two factor matrices. We further extend the equivalence to multiple factor matrices and show that all the factor norms can be convex and smooth for any $p>0$. In contrast, the original Schatten-$p$ norm for $0<p<1$ is non-convex and non-smooth. As an example we conduct experiments on matrix completion. To utilize the convexity of the factor matrix norms, we adopt the accelerated proximal alternating linearized minimization algorithm and establish its sequence convergence. Experiments on both synthetic and real datasets exhibit its superior performance over the state-of-the-art methods. Its speed is also highly competitive.