IGL-Bench: Establishing the Comprehensive Benchmark for Imbalanced Graph Learning

TL;DR

This paper introduces IGL-Bench, a comprehensive benchmark for evaluating imbalanced graph learning algorithms across diverse datasets and tasks, addressing the lack of standardized evaluation protocols in the field.

Contribution

The paper presents IGL-Bench, the first unified benchmark with consistent protocols for fair comparison of 24 IGL algorithms on 16 datasets, covering node and graph tasks.

Findings

IGL algorithms improve performance under various imbalance conditions.

Robustness and efficiency vary significantly among algorithms.

Benchmark facilitates reproducible and fair evaluation of IGL methods.

Abstract

Deep graph learning has gained grand popularity over the past years due to its versatility and success in representing graph data across a wide range of domains. However, the pervasive issue of imbalanced graph data distributions, where certain parts exhibit disproportionally abundant data while others remain sparse, undermines the efficacy of conventional graph learning algorithms, leading to biased outcomes. To address this challenge, Imbalanced Graph Learning (IGL) has garnered substantial attention, enabling more balanced data distributions and better task performance. Despite the proliferation of IGL algorithms, the absence of consistent experimental protocols and fair performance comparisons pose a significant barrier to comprehending advancements in this field. To bridge this gap, we introduce IGL-Bench, a foundational comprehensive benchmark for imbalanced graph learning, embarking on 16 diverse graph datasets and 24 distinct IGL algorithms with uniform data processing and splitting strategies. Specifically, IGL-Bench systematically investigates state-of-the-art IGL algorithms in terms of effectiveness, robustness, and efficiency on node-level and graph-level tasks, with the scope of class-imbalance and topology-imbalance. Extensive experiments demonstrate the potential benefits of IGL algorithms on various imbalanced conditions, offering insights and opportunities in the IGL field. Further, we have developed an open-sourced and unified package to facilitate reproducible evaluation and inspire further innovative research, which is available at https://github.com/RingBDStack/IGL-Bench.