GraphiT: Efficient Node Classification on Text-Attributed Graphs with Prompt Optimized LLMs

TL;DR

GraphiT introduces a method to encode text-attributed graphs into concise textual prompts and optimizes LLM prompts automatically, significantly improving node classification performance while reducing manual effort and computational costs.

Contribution

This paper presents a novel framework for encoding graph data into text and automating prompt optimization for LLMs, enhancing efficiency and reproducibility in graph-based node classification.

Findings

Outperforms baseline LLM methods on three datasets

Automated prompt optimization improves accuracy without manual tuning

Graph encoding is cost-effective and uses fewer tokens

Abstract

The application of large language models (LLMs) to graph data has attracted a lot of attention recently. LLMs allow us to use deep contextual embeddings from pretrained models in text-attributed graphs, where shallow embeddings are often used for the text attributes of nodes. However, it is still challenging to efficiently encode the graph structure and features into a sequential form for use by LLMs. In addition, the performance of an LLM alone, is highly dependent on the structure of the input prompt, which limits their effectiveness as a reliable approach and often requires iterative manual adjustments that could be slow, tedious and difficult to replicate programmatically. In this paper, we propose GraphiT (Graphs in Text), a framework for encoding graphs into a textual format and optimizing LLM prompts for graph prediction tasks. Here we focus on node classification for text-attributed graphs. We encode the graph data for every node and its neighborhood into a concise text to enable LLMs to better utilize the information in the graph. We then further programmatically optimize the LLM prompts using the DSPy framework to automate this step and make it more efficient and reproducible. GraphiT outperforms our LLM-based baselines on three datasets and we show how the optimization step in GraphiT leads to measurably better results without manual prompt tweaking. We also demonstrated that our graph encoding approach is competitive to other graph encoding methods while being less expensive because it uses significantly less tokens for the same task.