Negative curvature obstructs acceleration for strongly geodesically convex optimization, even with exact first-order oracles

TL;DR

This paper proves that acceleration in Riemannian gradient descent is impossible in hyperbolic spaces and similar manifolds, establishing the optimality of gradient descent in these settings despite the potential for more advanced algorithms.

Contribution

It extends previous results by showing that even with exact gradient information, acceleration cannot be achieved in geodesically convex optimization on Hadamard manifolds.

Findings

Acceleration is impossible for deterministic algorithms with exact information.

Gradient descent is optimal for strongly geodesically convex functions in hyperbolic spaces.

The results apply to a broad class of Hadamard manifolds, including hyperbolic spaces and positive definite matrices.

Abstract

Hamilton and Moitra (2021) showed that, in certain regimes, it is not possible to accelerate Riemannian gradient descent in the hyperbolic plane if we restrict ourselves to algorithms which make queries in a (large) bounded domain and which receive gradients and function values corrupted by a (small) amount of noise. We show that acceleration remains unachievable for any deterministic algorithm which receives exact gradient and function-value information (unbounded queries, no noise). Our results hold for the classes of strongly and nonstrongly geodesically convex functions, and for a large class of Hadamard manifolds including hyperbolic spaces and the symmetric space $\mathrm{SL}(n) / \mathrm{SO}(n)$ of positive definite $n \times n$ matrices of determinant one. This cements a surprising gap between the complexity of convex optimization and geodesically convex optimization: for hyperbolic spaces, Riemannian gradient descent is optimal on the class of smooth and and strongly geodesically convex functions, in the regime where the condition number scales with the radius of the optimization domain. The key idea for proving the lower bound consists of perturbing the hard functions of Hamilton and Moitra (2021) with sums of bump functions chosen by a resisting oracle.