Riemannian accelerated gradient methods via extrapolation

TL;DR

This paper introduces a simple extrapolation-based acceleration scheme for Riemannian gradient methods, achieving optimal convergence rates and outperforming existing Riemannian Nesterov methods in practice.

Contribution

It presents a new acceleration technique for Riemannian gradient methods that is simpler and more computationally efficient than previous approaches.

Findings

Achieves asymptotically optimal convergence rates.

Demonstrates practical benefits through experiments.

Outperforms existing Riemannian Nesterov methods.

Abstract

In this paper, we propose a simple acceleration scheme for Riemannian gradient methods by extrapolating iterates on manifolds. We show when the iterates are generated from Riemannian gradient descent method, the accelerated scheme achieves the optimal convergence rate asymptotically and is computationally more favorable than the recently proposed Riemannian Nesterov accelerated gradient methods. Our experiments verify the practical benefit of the novel acceleration strategy.