Global rates of convergence for nonconvex optimization on manifolds

TL;DR

This paper establishes the first deterministic global convergence rates for Riemannian gradient descent and trust-region methods in nonconvex optimization on manifolds, matching unconstrained rates and requiring no initialization assumptions.

Contribution

It provides the first deterministic global convergence rates for nonconvex optimization algorithms on manifolds, extending classical results to Riemannian settings.

Findings

Gradient descent finds approximate first-order points in O(1/ε²) iterations.

Trust-region method finds approximate second-order points in O(1/ε³) iterations.

Results apply to optimization on compact submanifolds of Euclidean space.

Abstract

We consider the minimization of a cost function $f$ on a manifold $M$ using Riemannian gradient descent and Riemannian trust regions (RTR). We focus on satisfying necessary optimality conditions within a tolerance $\varepsilon$. Specifically, we show that, under Lipschitz-type assumptions on the pullbacks of $f$ to the tangent spaces of $M$, both of these algorithms produce points with Riemannian gradient smaller than $\varepsilon$ in $O(1/\varepsilon^2)$ iterations. Furthermore, RTR returns a point where also the Riemannian Hessian's least eigenvalue is larger than $-\varepsilon$ in $O(1/\varepsilon^3)$ iterations. There are no assumptions on initialization. The rates match their (sharp) unconstrained counterparts as a function of the accuracy $\varepsilon$ (up to constants) and hence are sharp in that sense. These are the first deterministic results for global rates of convergence to approximate first- and second-order Karush-Kuhn-Tucker points on manifolds. They apply in particular for optimization constrained to compact submanifolds of $\mathbb{R}^n$, under simpler assumptions.