LoReC: Rethinking Large Language Models for Graph Data Analysis

TL;DR

LoReC is a novel method that enhances Large Language Models' ability to analyze graph data, significantly improving performance over existing GraphLLM approaches and GNNs.

Contribution

We introduce LoReC, a plug-and-play framework that improves LLMs' understanding of graph data through three key stages, addressing current limitations.

Findings

LoReC outperforms existing GraphLLM methods across multiple datasets.

LoReC surpasses traditional GNN-based approaches in graph analysis tasks.

Extensive experiments validate the effectiveness of LoReC.

Abstract

The advent of Large Language Models (LLMs) has fundamentally reshaped the way we interact with graphs, giving rise to a new paradigm called GraphLLM. As revealed in recent studies, graph learning can benefit from LLMs. However, we observe limited benefits when we directly utilize LLMs to make predictions for graph-related tasks within GraphLLM paradigm, which even yields suboptimal results compared to conventional GNN-based approaches. Through in-depth analysis, we find this failure can be attributed to LLMs' limited capability for processing graph data and their tendency to overlook graph information. To address this issue, we propose LoReC (Look, Remember, and Contrast), a novel plug-and-play method for GraphLLM paradigm, which enhances LLM's understanding of graph data through three stages: (1) Look: redistributing attention to graph; (2) Remember: re-injecting graph information into the Feed-Forward Network (FFN); (3) Contrast: rectifying the vanilla logits produced in the decoding process. Extensive experiments demonstrate that LoReC brings notable improvements over current GraphLLM methods and outperforms GNN-based approaches across diverse datasets. The implementation is available at https://github.com/Git-King-Zhan/LoReC.