Inexact Riemannian Gradient Descent Method for Nonconvex Optimization

TL;DR

This paper introduces a novel inexact Riemannian gradient descent method for nonconvex optimization problems, providing convergence guarantees and demonstrating effectiveness in applications like matrix completion and PCA.

Contribution

It develops the first inexact gradient conditions on Riemannian manifolds and proves strong convergence results for the proposed inexact gradient descent algorithm.

Findings

Convergence guarantees under Riemannian Kurdyka-ojasiewicz property

Effective in applications like low-rank matrix completion and PCA

Encompasses Riemannian sharpness-aware minimization and extragradient methods

Abstract

Gradient descent methods are fundamental first-order optimization algorithms in both Euclidean spaces and Riemannian manifolds. However, the exact gradient is not readily available in many scenarios. This paper proposes a novel inexact Riemannian gradient descent algorithm for nonconvex problems, accompanied by a convergence guarantee. In particular, we establish two inexact gradient conditions on Riemannian manifolds for the first time, enabling precise gradient approximations. Our method demonstrates strong convergence results for both gradient sequences and function values. The global convergence with constructive convergence rates for the sequence of iterates is ensured under the Riemannian Kurdyka-\L ojasiewicz property. Furthermore, our algorithm encompasses two specific applications: Riemannian sharpness-aware minimization and Riemannian extragradient algorithm, both of which inherit the global convergence properties of the inexact gradient methods. Numerical experiments on low-rank matrix completion and principal component analysis problems validate the efficiency and practical relevance of the proposed approaches.