Enhancing Graph Neural Networks with Limited Labeled Data by Actively Distilling Knowledge from Large Language Models

TL;DR

This paper introduces a novel method combining Large Language Models and Graph Neural Networks through active learning to improve node classification in graphs with limited labeled data, outperforming existing approaches.

Contribution

It presents a new Graph-LLM-based active learning framework that leverages zero-shot reasoning of LLMs to enhance GNN performance with scarce labels.

Findings

Significant improvement in node classification accuracy with limited labels.

Outperforms state-of-the-art baselines by large margins.

Effective integration of LLMs and GNNs for few-shot learning.

Abstract

Graphs are pervasive in the real-world, such as social network analysis, bioinformatics, and knowledge graphs. Graph neural networks (GNNs) have great ability in node classification, a fundamental task on graphs. Unfortunately, conventional GNNs still face challenges in scenarios with few labeled nodes, despite the prevalence of few-shot node classification tasks in real-world applications. To address this challenge, various approaches have been proposed, including graph meta-learning, transfer learning, and methods based on Large Language Models (LLMs). However, traditional meta-learning and transfer learning methods often require prior knowledge from base classes or fail to exploit the potential advantages of unlabeled nodes. Meanwhile, LLM-based methods may overlook the zero-shot capabilities of LLMs and rely heavily on the quality of generated contexts. In this paper, we propose a novel approach that integrates LLMs and GNNs, leveraging the zero-shot inference and reasoning capabilities of LLMs and employing a Graph-LLM-based active learning paradigm to enhance GNNs' performance. Extensive experiments demonstrate the effectiveness of our model in improving node classification accuracy with considerably limited labeled data, surpassing state-of-the-art baselines by significant margins.