GNN-as-Judge: Unleashing the Power of LLMs for Graph Learning with GNN Feedback

TL;DR

This paper introduces GNN-as-Judge, a framework that combines LLMs and GNNs to improve semi-supervised learning on text-attributed graphs with limited labels.

Contribution

It proposes a collaborative pseudo-labeling strategy and a weakly-supervised fine-tuning algorithm to enhance LLM performance in low-resource graph settings.

Findings

GNN-as-Judge outperforms existing methods on multiple TAG datasets.

The framework is especially effective in low-resource, few-shot scenarios.

It effectively mitigates label noise during LLM fine-tuning.

Abstract

Large Language Models (LLMs) have shown strong performance on text-attributed graphs (TAGs) due to their superior semantic understanding ability on textual node features. However, their effectiveness as predictors in the low-resource setting, where labeled nodes are severely limited and scarce, remains constrained since fine-tuning LLMs usually requires sufficient labeled data, especially when the TAG shows complex structural patterns. In essence, this paper targets two key challenges: (i) the difficulty of generating and selecting reliable pseudo labels on TAGs for LLMs, and (ii) the need to mitigate potential label noise when fine-tuning LLMs with pseudo labels. To counter the challenges, we propose a new framework, GNN-as-Judge, which can unleash the power of LLMs for few-shot semi-supervised learning on TAGs by incorporating the structural inductive bias of Graph Neural Networks (GNNs). Specifically, GNN-as-Judge introduces a collaborative pseudo-labeling strategy that first identifies the most influenced unlabeled nodes from labeled nodes, then exploits both the agreement and disagreement patterns between LLMs and GNNs to generate reliable labels. Furthermore, we develop a weakly-supervised LLM fine-tuning algorithm that can distill the knowledge from informative pseudo labels while mitigating the potential label noise. Experiments on multiple TAG datasets demonstrate that GNN-as-Judge significantly outperforms existing methods, particularly in low-resource regimes where labeled data are scarce.