Dynamic Proximal Gradient Algorithms for Schatten-$p$ Quasi-Norm Regularized Problems

TL;DR

This paper introduces a dynamic proximal gradient algorithm for Schatten-$p$ quasi-norm regularized problems that reduces computational complexity and guarantees convergence, with demonstrated efficiency in numerical experiments.

Contribution

The paper proposes a novel algorithm using Cayley transformation to efficiently solve Schatten-$p$ regularized problems, with proven convergence properties and improved computational performance.

Findings

The algorithm avoids expensive SVD computations at each iteration.

It achieves sublinear convergence for all p in [0,1].

Numerical results show superior computational efficiency.

Abstract

This paper investigates numerical solution methods for the Schatten-$p$ quasi-norm regularized problem with $p \in [0,1]$, which has been widely studied for finding low-rank solutions of linear inverse problems and gained successful applications in various mathematics and applied science fields. We propose a dynamic proximal gradient algorithm that, through the use of the Cayley transformation, avoids computationally expensive singular value decompositions at each iteration, thereby significantly reducing the computational complexity. The algorithm incorporates two step size selection strategies: an adaptive backtracking search and an explicit step size rule. We establish the sublinear convergence of the proposed algorithm for all $p \in [0,1]$ within the framework of the Kurdyka-Lojasiewicz property. Notably, under mild assumptions, we show that the generated sequence converges to a stationary point of the objective function of the problem. For the special case when $p=1$, the linear convergence is further proved under the strict complementarity-type regularity condition commonly used in the linear convergence analysis of the forward-backward splitting algorithms. Preliminary numerical results validate the superior computational efficiency of the proposed algorithm.