Language is All a Graph Needs

TL;DR

This paper introduces InstructGLM, a novel approach that uses instruction-finetuned large language models to perform graph learning tasks, surpassing traditional GNNs on multiple datasets by leveraging natural language descriptions of graph structures.

Contribution

It presents a new generative graph learning framework using LLMs with natural language instructions, replacing GNNs as a foundation model for graph analysis.

Findings

Outperforms GNN baselines on ogbn-arxiv, Cora, and PubMed datasets.

Uses natural language to describe multi-scale graph structures.

Demonstrates effectiveness of generative LLMs for graph tasks.

Abstract

The emergence of large-scale pre-trained language models has revolutionized various AI research domains. Transformers-based Large Language Models (LLMs) have gradually replaced CNNs and RNNs to unify fields of computer vision and natural language processing. Compared with independent data samples such as images, videos or texts, graphs usually contain rich structural and relational information. Meanwhile, language, especially natural language, being one of the most expressive mediums, excels in describing complex structures. However, existing work on incorporating graph problems into the generative language modeling framework remains very limited. Considering the rising prominence of LLMs, it becomes essential to explore whether LLMs can also replace GNNs as the foundation model for graphs. In this paper, we propose InstructGLM (Instruction-finetuned Graph Language Model) with highly scalable prompts based on natural language instructions. We use natural language to describe multi-scale geometric structure of the graph and then instruction finetune an LLM to perform graph tasks, which enables Generative Graph Learning. Our method surpasses all GNN baselines on ogbn-arxiv, Cora and PubMed datasets, underscoring its effectiveness and sheds light on generative LLMs as new foundation model for graph machine learning. Our code is open-sourced at https://github.com/agiresearch/InstructGLM.