Riemannian Gradient Method with Momentum

TL;DR

This paper introduces a Riemannian gradient method with momentum that extends existing unconstrained optimization techniques to manifold settings, providing theoretical guarantees and demonstrating superior performance through computational experiments.

Contribution

It presents a novel Riemannian gradient method with momentum, including a safeguarding rule and complexity analysis, extending recent unconstrained optimization methods to Riemannian manifolds.

Findings

Achieves an $oldsymbol{ ext{O}( ext{epsilon}^{-2})}$ complexity bound.

Demonstrates superior performance over state-of-the-art solvers in experiments.

Proves robustness and effectiveness of the proposed method.

Abstract

In this paper, we consider the problem of minimizing a smooth function on a Riemannian manifold and present a Riemannian gradient method with momentum. The proposed algorithm represents a substantial and nontrivial extension of a recently introduced method for unconstrained optimization. We prove that the algorithm, supported by a safeguarding rule, produces an $\epsilon$-stationary point with a worst-case complexity bound of $\mathcal{O}(\epsilon^{-2})$. Extensive computational experiments on benchmark problems are carried out, comparing the proposed method with state-of-the-art solvers available in the Manopt package. The results demonstrate competitive and often superior performance. Overall, the numerical evidence confirms the effectiveness and robustness of the proposed approach, which provides a meaningful extension of the recently introduced momentum-based method to Riemannian optimization.