No-go Theorem for Acceleration in the Hyperbolic Plane

TL;DR

This paper proves that accelerated gradient methods akin to Nesterov's are impossible for geodesically convex functions in the hyperbolic plane, especially under noisy conditions, due to the space's negative curvature affecting information retention.

Contribution

It establishes a no-go theorem showing the non-existence of Nesterov-like acceleration in hyperbolic geometry for convex optimization.

Findings

No accelerated gradient descent in hyperbolic plane under noise

Negative curvature causes rapid volume growth, hindering acceleration

Results hold even with exponentially small noise

Abstract

In recent years there has been significant effort to adapt the key tools and ideas in convex optimization to the Riemannian setting. One key challenge has remained: Is there a Nesterov-like accelerated gradient method for geodesically convex functions on a Riemannian manifold? Recent work has given partial answers and the hope was that this ought to be possible. Here we dash these hopes. We prove that in a noisy setting, there is no analogue of accelerated gradient descent for geodesically convex functions on the hyperbolic plane. Our results apply even when the noise is exponentially small. The key intuition behind our proof is short and simple: In negatively curved spaces, the volume of a ball grows so fast that information about the past gradients is not useful in the future.