Klein Model for Hyperbolic Neural Networks

TL;DR

This paper introduces a Klein model-based framework for hyperbolic neural networks, leveraging its geometric advantages and demonstrating comparable performance to existing models like the Poincaré ball.

Contribution

It presents the first detailed formulation of Klein model operations for HNNs and proves their equivalence to Einstein scalar multiplication and addition.

Findings

Klein HNN performs on par with Poincaré ball HNN.

Provides detailed formulations for Klein model operations.

Establishes Klein linear layer and geometric operations.

Abstract

Hyperbolic neural networks (HNNs) have been proved effective in modeling complex data structures. However, previous works mainly focused on the Poincar\'e ball model and the hyperboloid model as coordinate representations of the hyperbolic space, often neglecting the Klein model. Despite this, the Klein model offers its distinct advantages thanks to its straight-line geodesics, which facilitates the well-known Einstein midpoint construction, previously leveraged to accompany HNNs in other models. In this work, we introduce a framework for hyperbolic neural networks based on the Klein model. We provide detailed formulation for representing useful operations using the Klein model. We further study the Klein linear layer and prove that the "tangent space construction" of the scalar multiplication and parallel transport are exactly the Einstein scalar multiplication and the Einstein addition, analogous to the M\"obius operations used in the Poincar\'e ball model. We show numerically that the Klein HNN performs on par with the Poincar\'e ball model, providing a third option for HNN that works as a building block for more complicated architectures.