LangTopo: Aligning Language Descriptions of Graphs with Tokenized Topological Modeling

TL;DR

LangTopo is a novel framework that aligns language descriptions with graph topological structures at the token level, enabling LLMs to independently understand and process graph-structured data.

Contribution

It introduces a codebook-based alignment method that bridges LLMs and GNNs for better graph structure modeling from natural language descriptions.

Findings

LangTopo improves LLMs' ability to model graph structures.

The framework achieves superior performance on multiple graph datasets.

It effectively enables LLMs to handle graph data independently.

Abstract

Recently, large language models (LLMs) have been widely researched in the field of graph machine learning due to their outstanding abilities in language comprehension and learning. However, the significant gap between natural language tasks and topological structure modeling poses a nonnegligible challenge. Specifically, since natural language descriptions are not sufficient for LLMs to understand and process graph-structured data, fine-tuned LLMs perform even worse than some traditional GNN models on graph tasks, lacking inherent modeling capabilities for graph structures. Existing research overly emphasizes LLMs' understanding of semantic information captured by external models, while inadequately exploring graph topological structure modeling, thereby overlooking the genuine capabilities that LLMs lack. Consequently, in this paper, we introduce a new framework, LangTopo, which aligns graph structure modeling with natural language understanding at the token level. LangTopo quantifies the graph structure modeling capabilities of GNNs and LLMs by constructing a codebook for the graph modality and performs consistency maximization. This process aligns the text description of LLM with the topological modeling of GNN, allowing LLM to learn the ability of GNN to capture graph structures, enabling LLM to handle graph-structured data independently. We demonstrate the effectiveness of our proposed method on multiple datasets.