R-SPIDER: A Fast Riemannian Stochastic Optimization Algorithm with Curvature Independent Rate

TL;DR

This paper introduces R-SPIDER, a Riemannian stochastic optimization algorithm that achieves faster convergence rates without depending on manifold curvature, applicable to both nonconvex and strongly convex problems.

Contribution

It adapts the SPIDER variance reduction method to Riemannian manifolds, providing curvature-independent convergence rates unlike previous approaches.

Findings

Achieves faster convergence rates than existing algorithms.

Provides curvature-independent analysis for nonconvex and strongly convex cases.

Applicable to both finite sum and stochastic optimization settings.

Abstract

We study smooth stochastic optimization problems on Riemannian manifolds. Via adapting the recently proposed SPIDER algorithm \citep{fang2018spider} (a variance reduced stochastic method) to Riemannian manifold, we can achieve faster rate than known algorithms in both the finite sum and stochastic settings. Unlike previous works, by \emph{not} resorting to bounding iterate distances, our analysis yields curvature independent convergence rates for both the nonconvex and strongly convex cases.