GraphBench: Next-generation graph learning benchmarking

TL;DR

GraphBench is a comprehensive benchmarking suite for graph learning that standardizes evaluation protocols across diverse tasks and domains, addressing fragmentation and reproducibility issues.

Contribution

It introduces a unified benchmark with standardized datasets, splits, metrics, and hyperparameter tuning for evaluating graph models.

Findings

Established baseline performances for recent graph models.

Demonstrated the importance of standardized evaluation for progress.

Provided a platform for future graph learning research.

Abstract

Machine learning on graphs has made substantial progress across domains such as molecular property prediction and chip design. Yet benchmarking practices remain fragmented, often relying on narrow, task-specific datasets and inconsistent evaluation protocols, hindering reproducibility and broader progress. With the recent popularity of graph foundation models, these weaknesses have become apparent, as existing benchmarks are insufficient for thorough evaluation. To address these challenges, we introduce GraphBench, a comprehensive benchmark suite spanning diverse real-world domains and task settings, including node-level, edge-level, graph-level, and generative tasks. GraphBench provides standardized evaluation protocols, including consistent dataset splits and metrics for assessing out-of-distribution generalization across selected tasks, as well as a unified hyperparameter-tuning framework. We further evaluate GraphBench with recent message-passing neural networks and graph transformer models, establishing principled baselines for future research. See www.graphbench.io for further details.