Scalable Nuclear-norm Minimization by Subspace Pursuit Proximal Riemannian Gradient

TL;DR

This paper introduces a scalable optimization framework combining proximal Riemannian gradient and subspace pursuit for trace-norm regularized problems, significantly reducing computational costs in low-rank matrix tasks.

Contribution

It proposes a novel subspace pursuit paradigm integrated with PRG to efficiently solve trace-norm regularized problems without explicit rank constraints.

Findings

Outperforms existing methods in matrix completion tasks.

Reduces computational complexity by avoiding large-rank SVDs.

Demonstrates effectiveness in subspace clustering applications.

Abstract

Nuclear-norm regularization plays a vital role in many learning tasks, such as low-rank matrix recovery (MR), and low-rank representation (LRR). Solving this problem directly can be computationally expensive due to the unknown rank of variables or large-rank singular value decompositions (SVDs). To address this, we propose a proximal Riemannian gradient (PRG) scheme which can efficiently solve trace-norm regularized problems defined on real-algebraic variety $\mMLr$ of real matrices of rank at most $r$. Based on PRG, we further present a simple and novel subspace pursuit (SP) paradigm for general trace-norm regularized problems without the explicit rank constraint $\mMLr$. The proposed paradigm is very scalable by avoiding large-rank SVDs. Empirical studies on several tasks, such as matrix completion and LRR based subspace clustering, demonstrate the superiority of the proposed paradigms over existing methods.