Position: How can Graphs Help Large Language Models?

TL;DR

This paper explores how integrating graphs with large language models can improve their reasoning, reduce hallucinations, and enhance understanding of structured data across various domains.

Contribution

It introduces graph-based prompting techniques like GoT and discusses how graphs can serve as up-to-date knowledge sources to aid LLMs.

Findings

Graph-based prompting techniques enhance LLM reasoning.

Graphs help reduce LLM hallucinations by providing current knowledge.

Integrating graphs improves LLM understanding of structured data.

Abstract

With the rapid advancement of large language models (LLMs), classic graph learning tasks have greatly benefited from LLMs, including improved encoding of textual features, more efficient construction of graphs from text, and enhanced reasoning over knowledge graphs. In this paper, we ask a complementary question: How can graphs help LLMs? We address this question from three perspectives: 1) graphs provide an up-to-date knowledge source that helps reduce LLM hallucinations, 2) graph-based prompting techniques-such as Chain-of-Thought (CoT), Tree-of-Thought (ToT), and Graph-of-Thought (GoT)-enhance LLM reasoning capabilities, and 3) integrating graphs into LLMs improves their understanding of structured data, expanding their applicability to domains such as e-commerce, code, and relational databases (RDBs). We further outlook some future directions including designing sparse LLM architectures based on graphs and brain-inspired memory systems.