CodeGraph: Enhancing Graph Reasoning of LLMs with Code
Qiaolong Cai, Zhaowei Wang, Shizhe Diao, James Kwok, Yangqiu Song

TL;DR
CodeGraph encodes graph reasoning problems as executable code, significantly improving LLMs' accuracy and interpretability in solving complex graph algorithm tasks by learning from examples and executing generated programs.
Contribution
The paper introduces CodeGraph, a novel approach that encodes graph reasoning solutions as code, enhancing LLM performance and interpretability in graph algorithm problems.
Findings
Boosts LLM performance on graph reasoning tasks by up to 58.6%.
Improves accuracy on arithmetic problems in graph tasks.
Provides a more controllable and interpretable reasoning process.
Abstract
With the increasing popularity of large language models (LLMs), reasoning on basic graph algorithm problems is an essential intermediate step in assessing their abilities to process and infer complex graph reasoning tasks. Existing methods usually convert graph-structured data to textual descriptions and then use LLMs for reasoning and computation. However, LLMs often produce computation errors on arithmetic parts in basic graph algorithm problems, such as counting number of edges. In addition, they struggle to control or understand the output of the reasoning process, raising concerns about whether LLMs are simply guessing. In this paper, we introduce CodeGraph, a method that encodes graph problem solutions as code. The methods solve new graph problems by learning from exemplars, generating programs, and executing them via a program interpreter. Using the few-shot setting, we evaluate…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Wikis in Education and Collaboration
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · {Dispute@FaQ-s}How to file a dispute with Expedia? · 15 Ways to Contact How can i speak to someone at Delta Airlines · Adam · Layer Normalization · Weight Decay · Attention Is All You Need · Dense Connections · Attention Dropout · Cosine Annealing
