Hyperbolic Representation Learning: Revisiting and Advancing

TL;DR

This paper critically examines hyperbolic representation learning, introduces a position-tracking mechanism to evaluate existing methods, and proposes a simple, effective hyperbolic informed embedding (HIE) that significantly improves performance across tasks.

Contribution

It introduces a position-tracking mechanism to evaluate hyperbolic models and proposes HIE, a task- and model-agnostic method leveraging hyperbolic distance for better hierarchical embedding.

Findings

Existing hyperbolic models are sub-optimal according to position-tracking.

HIE improves performance by up to 21.4% over baselines.

HIE is versatile and applicable across various models and tasks.

Abstract

The non-Euclidean geometry of hyperbolic spaces has recently garnered considerable attention in the realm of representation learning. Current endeavors in hyperbolic representation largely presuppose that the underlying hierarchies can be automatically inferred and preserved through the adaptive optimization process. This assumption, however, is questionable and requires further validation. In this work, we first introduce a position-tracking mechanism to scrutinize existing prevalent \hlms, revealing that the learned representations are sub-optimal and unsatisfactory. To address this, we propose a simple yet effective method, hyperbolic informed embedding (HIE), by incorporating cost-free hierarchical information deduced from the hyperbolic distance of the node to origin (i.e., induced hyperbolic norm) to advance existing \hlms. The proposed method HIE is both task-agnostic and model-agnostic, enabling its seamless integration with a broad spectrum of models and tasks. Extensive experiments across various models and different tasks demonstrate the versatility and adaptability of the proposed method. Remarkably, our method achieves a remarkable improvement of up to 21.4\% compared to the competing baselines.