Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein space

TL;DR

This paper introduces a novel Riemannian optimization algorithm with dynamic stepsize acceleration, applicable to Wasserstein space and symmetric positive definite matrices, advancing the theory and practice of optimization on manifolds.

Contribution

It develops a new class of Riemannian algorithms with dynamic stepsize acceleration, including the first provably accelerated gradient method in Wasserstein space.

Findings

Recovers standard Wasserstein gradient descent.

Provides the first provable acceleration in Wasserstein space.

Demonstrates numerical effectiveness on benchmark tasks.

Abstract

There is extensive literature on accelerating first-order optimization methods in a Euclidean setting. Under which conditions such acceleration is feasible in Riemannian optimization problems is an active area of research. Motivated by the recent success of dynamic stepsize methods in the Euclidean setting, we undertake a study of such algorithms in the Riemannian setting. We provide the new class of algorithms determined by the choice of vector transport that allows the dynamic stepsize acceleration on Riemannian manifolds for the function classes associated with the corresponding vector transport. As a core application, we show our algorithm recovers the standard Wasserstein gradient descent on 2-Wasserstein space, and as a result provides the first provable accelerated gradient method in Wasserstein space. In addition, we validate the numerical strength of the algorithm for standard benchmark tasks on the space of symmetric positive definite matrices.