Improving Hyperbolic Representations via Gromov-Wasserstein Regularization

TL;DR

This paper introduces a novel regularization method using Gromov-Wasserstein distance in hyperbolic neural networks to better preserve data structures, improving performance in hierarchical data modeling tasks.

Contribution

It proposes using Gromov-Wasserstein distance as a regularizer in hyperbolic neural networks, explicitly treating network layers as transport maps to maintain data geometry.

Findings

Enhanced data structure preservation in hyperbolic embeddings

Improved accuracy in few-shot image classification

Better semi-supervised graph link prediction and node classification

Abstract

Hyperbolic representations have shown remarkable efficacy in modeling inherent hierarchies and complexities within data structures. Hyperbolic neural networks have been commonly applied for learning such representations from data, but they often fall short in preserving the geometric structures of the original feature spaces. In response to this challenge, our work applies the Gromov-Wasserstein (GW) distance as a novel regularization mechanism within hyperbolic neural networks. The GW distance quantifies how well the original data structure is maintained after embedding the data in a hyperbolic space. Specifically, we explicitly treat the layers of the hyperbolic neural networks as a transport map and calculate the GW distance accordingly. We validate that the GW distance computed based on a training set well approximates the GW distance of the underlying data distribution. Our approach demonstrates consistent enhancements over current state-of-the-art methods across various tasks, including few-shot image classification, as well as semi-supervised graph link prediction and node classification.