Let's Ask GNN: Empowering Large Language Model for Graph In-Context Learning

TL;DR

AskGNN is a novel method that enhances large language models' ability to perform graph-related tasks by integrating graph neural networks with in-context learning, enabling effective use of graph data without extensive fine-tuning.

Contribution

This paper introduces AskGNN, a GNN-powered retriever that improves LLMs' performance on graph tasks through optimized in-context learning with complex graph structures.

Findings

AskGNN outperforms baseline methods on multiple graph tasks.

The approach works effectively across various LLMs.

It reduces the need for extensive fine-tuning on graph data.

Abstract

Textual Attributed Graphs (TAGs) are crucial for modeling complex real-world systems, yet leveraging large language models (LLMs) for TAGs presents unique challenges due to the gap between sequential text processing and graph-structured data. We introduce AskGNN, a novel approach that bridges this gap by leveraging In-Context Learning (ICL) to integrate graph data and task-specific information into LLMs. AskGNN employs a Graph Neural Network (GNN)-powered structure-enhanced retriever to select labeled nodes across graphs, incorporating complex graph structures and their supervision signals. Our learning-to-retrieve algorithm optimizes the retriever to select example nodes that maximize LLM performance on graph. Experiments across three tasks and seven LLMs demonstrate AskGNN's superior effectiveness in graph task performance, opening new avenues for applying LLMs to graph-structured data without extensive fine-tuning.