Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark

TL;DR

This paper introduces BeGIN, a comprehensive benchmark for evaluating GNNs under realistic, instance-dependent label noise conditions, highlighting the challenges and guiding future robustness improvements.

Contribution

It presents a new benchmark with realistic noise simulations, including LLM-based methods, and evaluates strategies for noise robustness in GNNs, addressing a gap in existing studies.

Findings

LLM-based corruption poses significant challenges for GNNs.

Node-specific parameterization improves robustness against label noise.

The benchmark enables systematic evaluation of noise-handling strategies.

Abstract

Graph Neural Networks (GNNs) have achieved state-of-the-art performance in node classification tasks but struggle with label noise in real-world data. Existing studies on graph learning with label noise commonly rely on class-dependent label noise, overlooking the complexities of instance-dependent noise and falling short of capturing real-world corruption patterns. We introduce BeGIN (Benchmarking for Graphs with Instance-dependent Noise), a new benchmark that provides realistic graph datasets with various noise types and comprehensively evaluates noise-handling strategies across GNN architectures, noisy label detection, and noise-robust learning. To simulate instance-dependent corruptions, BeGIN introduces algorithmic methods and LLM-based simulations. Our experiments reveal the challenges of instance-dependent noise, particularly LLM-based corruption, and underscore the importance of node-specific parameterization to enhance GNN robustness. By comprehensively evaluating noise-handling strategies, BeGIN provides insights into their effectiveness, efficiency, and key performance factors. We expect that BeGIN will serve as a valuable resource for advancing research on label noise in graphs and fostering the development of robust GNN training methods. The code is available at https://github.com/kimsu55/BeGIN.