NoisyGL: A Comprehensive Benchmark for Graph Neural Networks under Label Noise

TL;DR

NoisyGL is the first comprehensive benchmark designed to evaluate graph neural networks under label noise, enabling fair comparisons and revealing new insights to advance research in noisy graph data scenarios.

Contribution

This paper introduces NoisyGL, a unified benchmark for GNNs under label noise, facilitating consistent evaluation and deeper understanding of methods in this challenging setting.

Findings

Uncovered important insights missed in previous studies

Provided a unified experimental framework for fair comparison

Open-sourced benchmark library for future research

Abstract

Graph Neural Networks (GNNs) exhibit strong potential in node classification task through a message-passing mechanism. However, their performance often hinges on high-quality node labels, which are challenging to obtain in real-world scenarios due to unreliable sources or adversarial attacks. Consequently, label noise is common in real-world graph data, negatively impacting GNNs by propagating incorrect information during training. To address this issue, the study of Graph Neural Networks under Label Noise (GLN) has recently gained traction. However, due to variations in dataset selection, data splitting, and preprocessing techniques, the community currently lacks a comprehensive benchmark, which impedes deeper understanding and further development of GLN. To fill this gap, we introduce NoisyGL in this paper, the first comprehensive benchmark for graph neural networks under label noise. NoisyGL enables fair comparisons and detailed analyses of GLN methods on noisy labeled graph data across various datasets, with unified experimental settings and interface. Our benchmark has uncovered several important insights that were missed in previous research, and we believe these findings will be highly beneficial for future studies. We hope our open-source benchmark library will foster further advancements in this field. The code of the benchmark can be found in https://github.com/eaglelab-zju/NoisyGL.