DHG-Bench: A Comprehensive Benchmark for Deep Hypergraph Learning

TL;DR

DHG-Bench is the first comprehensive benchmark for deep hypergraph neural networks, systematically evaluating 17 algorithms across multiple dimensions and datasets to advance understanding and development in hypergraph learning.

Contribution

It introduces DHG-Bench, a unified benchmarking framework with extensive evaluation protocols, covering effectiveness, efficiency, robustness, and fairness of HNNs on diverse tasks and datasets.

Findings

Identified strengths and limitations of current HNN algorithms.

Provided insights into algorithm performance across various tasks.

Facilitated reproducible research with an open-source library.

Abstract

Deep graph models have achieved great success in network representation learning. However, their focus on pairwise relationships restricts their ability to learn pervasive higher-order interactions in real-world systems, which can be naturally modeled as hypergraphs. To tackle this issue, Hypergraph Neural Networks (HNNs) have garnered substantial attention in recent years. Despite the proposal of numerous HNNs, the absence of consistent experimental protocols and multi-dimensional empirical analysis impedes deeper understanding and further development of HNN research. While several toolkits for deep hypergraph learning (DHGL) have been introduced to facilitate algorithm evaluation, they provide only limited quantitative evaluation results and insufficient coverage of advanced algorithms, datasets, and benchmark tasks. To fill the gap, we introduce DHG-Bench, the first comprehensive benchmark for HNNs. Specifically, DHG-Bench systematically investigates the characteristics of HNNs in terms of four dimensions: effectiveness, efficiency, robustness, and fairness. We comprehensively evaluate 17 state-of-the-art HNN algorithms on 22 diverse datasets spanning node-, edge-, and graph-level tasks, under unified experimental settings. Extensive experiments reveal both the strengths and limitations of existing algorithms, offering valuable insights and directions for future research. Furthermore, to facilitate reproducible research, we have developed an easy-to-use library for training and evaluating different HNN methods. The DHG-Bench library is available at: https://github.com/Coco-Hut/DHG-Bench.