Hyperbolic Neural Networks

TL;DR

This paper develops hyperbolic neural network layers using M"obius gyrovector spaces and Riemannian geometry, enabling hyperbolic embeddings for downstream tasks and demonstrating competitive performance in NLP tasks.

Contribution

It introduces hyperbolic neural network layers based on M"obius gyrovector spaces, bridging the gap between hyperbolic geometry and deep learning tools.

Findings

Hyperbolic embeddings outperform Euclidean ones in certain NLP tasks.

The proposed hyperbolic neural networks are effective for sequential data classification.

Hyperbolic models achieve comparable or better results than Euclidean models.

Abstract

Hyperbolic spaces have recently gained momentum in the context of machine learning due to their high capacity and tree-likeliness properties. However, the representational power of hyperbolic geometry is not yet on par with Euclidean geometry, mostly because of the absence of corresponding hyperbolic neural network layers. This makes it hard to use hyperbolic embeddings in downstream tasks. Here, we bridge this gap in a principled manner by combining the formalism of M\"obius gyrovector spaces with the Riemannian geometry of the Poincar\'e model of hyperbolic spaces. As a result, we derive hyperbolic versions of important deep learning tools: multinomial logistic regression, feed-forward and recurrent neural networks such as gated recurrent units. This allows to embed sequential data and perform classification in the hyperbolic space. Empirically, we show that, even if hyperbolic optimization tools are limited, hyperbolic sentence embeddings either outperform or are on par with their Euclidean variants on textual entailment and noisy-prefix recognition tasks.