GUNDAM: Aligning Large Language Models with Graph Understanding

TL;DR

This paper introduces GUNDAM, a model that adapts large language models to better understand and reason about graph structures, significantly improving graph reasoning tasks beyond textual data.

Contribution

The paper presents GUNDAM, a novel approach that enhances LLMs' ability to comprehend and utilize graph structures, with theoretical analysis and superior performance on benchmarks.

Findings

GUNDAM outperforms state-of-the-art baselines in graph reasoning benchmarks.

Key factors influencing LLMs' graph reasoning capabilities are identified.

Theoretical analysis shows reasoning paths improve LLMs' reasoning abilities.

Abstract

Large Language Models (LLMs) have achieved impressive results in processing text data, which has sparked interest in applying these models beyond textual data, such as graphs. In the field of graph learning, there is a growing interest in harnessing LLMs to comprehend and manipulate graph-structured data. Existing research predominantly focuses on graphs with rich textual features, such as knowledge graphs or text attribute graphs, leveraging LLMs' ability to process text but inadequately addressing graph structure. This work specifically aims to assess and enhance LLMs' abilities to comprehend and utilize the structural knowledge inherent in graph data itself, rather than focusing solely on graphs rich in textual content. To achieve this, we introduce the \textbf{G}raph \textbf{U}nderstanding for \textbf{N}atural Language \textbf{D}riven \textbf{A}nalytical \textbf{M}odel (\model). This model adapts LLMs to better understand and engage with the structure of graph data, enabling them to perform complex reasoning tasks by leveraging the graph's structure itself. Our experimental evaluations on graph reasoning benchmarks not only substantiate that \model~ outperforms the SOTA baselines for comparisons. But also reveals key factors affecting the graph reasoning capabilities of LLMs. Moreover, we provide a theoretical analysis illustrating how reasoning paths can enhance LLMs' reasoning capabilities.