GCoder: Improving Large Language Model for Generalized Graph Problem Solving

TL;DR

GCoder is a novel code-based large language model that significantly improves generalized graph problem-solving capabilities, handling diverse formats and large-scale graphs more effectively than previous reasoning-based approaches.

Contribution

Introduces GCoder, a code-based LLM trained on a diverse graph dataset with multi-stage fine-tuning, enhancing generalization and scalability in graph problem-solving.

Findings

Outperforms GPT-4o with 16.42% accuracy gain across graph tasks.

Handles large graphs with millions of nodes efficiently.

Overcomes limitations of reasoning steps paradigm in graph computation.

Abstract

Large Language Models (LLMs) have demonstrated strong reasoning abilities, making them suitable for complex tasks such as graph computation. Traditional reasoning steps paradigm for graph problems is hindered by unverifiable steps, limited long-term reasoning, and poor generalization to graph variations. To overcome these limitations, we introduce GCoder, a code-based LLM designed to enhance problem-solving in generalized graph computation problems. Our method involves constructing an extensive training dataset, GraphWild, featuring diverse graph formats and algorithms. We employ a multi-stage training process, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Compiler Feedback (RLCF), to refine model capabilities. For unseen tasks, a hybrid retrieval technique is used to augment performance. Experiments demonstrate that GCoder outperforms GPT-4o, with an average accuracy improvement of 16.42% across various graph computational problems. Furthermore, GCoder efficiently manages large-scale graphs with millions of nodes and diverse input formats, overcoming the limitations of previous models focused on the reasoning steps paradigm. This advancement paves the way for more intuitive and effective graph problem-solving using LLMs. Code and data are available at here: https://github.com/Bklight999/WWW25-GCoder/tree/master.