Computing Bouligand stationary points efficiently in low-rank optimization

TL;DR

This paper introduces an efficient first-order algorithm for finding Bouligand stationary points in low-rank matrix optimization, reducing computational cost by requiring smaller SVDs and providing convergence guarantees.

Contribution

It proposes a novel first-order method that efficiently computes Bouligand stationary points with low-rank SVDs, improving over existing algorithms in computational complexity.

Findings

Convergence rate of O(1/√i) for Bouligand stationarity measure.

Algorithm requires SVDs of matrices with dimension at most r.

Rank-increasing scheme enhances the method when r is overestimated.

Abstract

This paper considers the problem of minimizing a differentiable function with locally Lipschitz continuous gradient on the algebraic variety of all $m$-by-$n$ real matrices of rank at most $r$. Several definitions of stationarity exist for this nonconvex problem. Among them, Bouligand stationarity is the strongest necessary condition for local optimality. Only a handful of algorithms generate a sequence in the variety whose accumulation points are provably Bouligand stationary. Among them, the most parsimonious with (truncated) singular value decompositions (SVDs) or eigenvalue decompositions can still require a truncated SVD of a matrix whose rank can be as large as $\min\{m, n\}-r+1$ if the gradient does not have low rank, which is computationally prohibitive in the typical case where $r \ll \min\{m, n\}$. This paper proposes a first-order algorithm that generates a sequence in the variety whose accumulation points are Bouligand stationary while requiring SVDs of matrices whose smaller dimension is always at most $r$. A standard measure of Bouligand stationarity converges to zero along the bounded subsequences at a rate at least $O(1/\sqrt{i+1})$, where $i$ is the iteration counter. Furthermore, a rank-increasing scheme based on the proposed algorithm is presented, which can be of interest if the parameter $r$ is potentially overestimated.