Projection-free nonconvex stochastic optimization on Riemannian manifolds

TL;DR

This paper introduces stochastic Riemannian Frank-Wolfe algorithms for constrained nonconvex optimization on manifolds, achieving competitive convergence rates and demonstrating state-of-the-art results in applications like Karcher means and Wasserstein barycenters.

Contribution

It develops the first stochastic projection-free methods for nonconvex Riemannian optimization, including variance reduction techniques like Riemannian Spider.

Findings

Achieves convergence rates comparable to Euclidean methods.

Demonstrates superior empirical performance on matrix mean and barycenter tasks.

Abstract

We study stochastic projection-free methods for constrained optimization of smooth functions on Riemannian manifolds, i.e., with additional constraints beyond the parameter domain being a manifold. Specifically, we introduce stochastic Riemannian Frank-Wolfe methods for nonconvex and geodesically convex problems. We present algorithms for both purely stochastic optimization and finite-sum problems. For the latter, we develop variance-reduced methods, including a Riemannian adaptation of the recently proposed Spider technique. For all settings, we recover convergence rates that are comparable to the best-known rates for their Euclidean counterparts. Finally, we discuss applications to two classic tasks: The computation of the Karcher mean of positive definite matrices and Wasserstein barycenters for multivariate normal distributions. For both tasks, stochastic Fw methods yield state-of-the-art empirical performance.