Gradient descent in hyperbolic space

TL;DR

This paper demonstrates that gradient descent in hyperbolic space, specifically using the hyperboloid model, is straightforward to implement and offers advantages for optimization tasks like computing the Fréchet mean.

Contribution

It clarifies the implementation of gradient descent in hyperbolic space and shows its benefits over approximation methods in this setting.

Findings

Gradient descent in hyperbolic space is simpler than previously thought.

Using the hyperboloid model facilitates straightforward calculations.

The approach improves optimization of the Fréchet mean in hyperbolic space.

Abstract

Gradient descent generalises naturally to Riemannian manifolds, and to hyperbolic $n$-space, in particular. Namely, having calculated the gradient at the point on the manifold representing the model parameters, the updated point is obtained by travelling along the geodesic passing in the direction of the gradient. Some recent works employing optimisation in hyperbolic space have not attempted this procedure, however, employing instead various approximations to avoid a calculation that was considered to be too complicated. In this tutorial, we demonstrate that in the hyperboloid model of hyperbolic space, the necessary calculations to perform gradient descent are in fact straight-forward. The advantages of the approach are then both illustrated and quantified for the optimisation problem of computing the Fr\'echet mean (i.e. barycentre) of points in hyperbolic space.