Shedding Light on Problems with Hyperbolic Graph Learning

TL;DR

This paper critically examines hyperbolic graph learning methods, revealing that simple Euclidean models often match or outperform hyperbolic ones, and identifies issues in current research practices such as lack of proper baselines and misleading metrics.

Contribution

The paper provides a thorough analysis of hyperbolic graph learning, highlights methodological flaws, and introduces a new benchmark dataset family for evaluating hyperbolic models.

Findings

Euclidean models perform comparably or better than hyperbolic models on many datasets.

Current hyperbolic methods often lack proper baselines and rely on misleading metrics.

A new benchmark dataset family is proposed to evaluate hyperbolic graph neural networks.

Abstract

Recent papers in the graph machine learning literature have introduced a number of approaches for hyperbolic representation learning. The asserted benefits are improved performance on a variety of graph tasks, node classification and link prediction included. Claims have also been made about the geometric suitability of particular hierarchical graph datasets to representation in hyperbolic space. Despite these claims, our work makes a surprising discovery: when simple Euclidean models with comparable numbers of parameters are properly trained in the same environment, in most cases, they perform as well, if not better, than all introduced hyperbolic graph representation learning models, even on graph datasets previously claimed to be the most hyperbolic as measured by Gromov $\delta$-hyperbolicity (i.e., perfect trees). This observation gives rise to a simple question: how can this be? We answer this question by taking a careful look at the field of hyperbolic graph representation learning as it stands today, and find that a number of results do not diligently present baselines, make faulty modelling assumptions when constructing algorithms, and use misleading metrics to quantify geometry of graph datasets. We take a closer look at each of these three problems, elucidate the issues, perform an analysis of methods, and introduce a parametric family of benchmark datasets to ascertain the applicability of (hyperbolic) graph neural networks.