Riemannian Optimization via Frank-Wolfe Methods

TL;DR

This paper introduces the Riemannian Frank-Wolfe (RFW) method for projection-free constrained optimization on manifolds, providing convergence analysis and practical algorithms for positive definite matrices and orthogonal groups.

Contribution

The paper proposes the RFW algorithm for Riemannian optimization, analyzes its convergence, and derives closed-form solutions for the linear oracle in specific manifolds, with applications to matrix means and synchronization.

Findings

RFW achieves non-asymptotic convergence rates for convex and nonconvex problems.

Closed-form solutions for the linear oracle are derived for positive definite matrices and orthogonal groups.

Empirical results show RFW performs competitively with state-of-the-art methods.

Abstract

We study projection-free methods for constrained Riemannian optimization. In particular, we propose the Riemannian Frank-Wolfe (RFW) method. We analyze non-asymptotic convergence rates of RFW to an optimum for (geodesically) convex problems, and to a critical point for nonconvex objectives. We also present a practical setting under which RFW can attain a linear convergence rate. As a concrete example, we specialize RFW to the manifold of positive definite matrices and apply it to two tasks: (i) computing the matrix geometric mean (Riemannian centroid); and (ii) computing the Bures-Wasserstein barycenter. Both tasks involve geodesically convex interval constraints, for which we show that the Riemannian "linear" oracle required by RFW admits a closed-form solution; this result may be of independent interest. We further specialize RFW to the special orthogonal group and show that here too, the Riemannian "linear" oracle can be solved in closed form. Here, we describe an application to the synchronization of data matrices (Procrustes problem). We complement our theoretical results with an empirical comparison of RFW against state-of-the-art Riemannian optimization methods and observe that RFW performs competitively on the task of computing Riemannian centroids.