Two Newton methods on the manifold of fixed-rank matrices endowed with Riemannian quotient geometries

TL;DR

This paper develops two Riemannian Newton methods for the manifold of fixed-rank matrices, leveraging quotient geometries to improve convergence analysis and computational efficiency for low-rank matrix problems.

Contribution

It introduces two novel Riemannian geometries on the fixed-rank matrix manifold and formulates corresponding Newton methods with enhanced convergence properties.

Findings

Methods exhibit faster convergence than Euclidean counterparts

Geometric framework simplifies convergence analysis

Computational overhead remains manageable

Abstract

We consider two Riemannian geometries for the manifold $\mathcal{M}(p,m\times n)$ of all $m\times n$ matrices of rank $p$. The geometries are induced on $\mathcal{M}(p,m\times n)$ by viewing it as the base manifold of the submersion $\pi:(M,N)\mapsto MN^T$, selecting an adequate Riemannian metric on the total space, and turning $\pi$ into a Riemannian submersion. The theory of Riemannian submersions, an important tool in Riemannian geometry, makes it possible to obtain expressions for fundamental geometric objects on $\mathcal{M}(p,m\times n)$ and to formulate the Riemannian Newton methods on $\mathcal{M}(p,m\times n)$ induced by these two geometries. The Riemannian Newton methods admit a stronger and more streamlined convergence analysis than the Euclidean counterpart, and the computational overhead due to the Riemannian geometric machinery is shown to be mild. Potential applications include low-rank matrix completion and other low-rank matrix approximation problems.