Quantizing Text-attributed Graphs for Semantic-Structural Integration

TL;DR

This paper introduces STAG, a self-supervised graph quantization framework that encodes structural information into discrete tokens, enabling effective zero-shot transfer learning with large language models for graph tasks.

Contribution

STAG presents a novel soft assignment and KL divergence guided quantization method for graph data, facilitating LLM integration without labeled data or manual graph verbalization.

Findings

Achieves state-of-the-art node classification results

Supports zero-shot transfer learning without labeled source data

Maintains compatibility across various LLM architectures

Abstract

Text-attributed graphs (TAGs) have emerged as a powerful representation for modeling complex relationships across diverse domains. With the rise of large language models (LLMs), there is growing interest in leveraging their capabilities for graph learning. However, current approaches face significant challenges in embedding structural information into LLM-compatible formats, requiring either computationally expensive alignment mechanisms or manual graph verbalization techniques that often lose critical structural details. Moreover, these methods typically require labeled data from source domains for effective transfer learning, significantly constraining their adaptability. We propose STAG, a novel self-supervised framework that directly quantizes graph structural information into discrete tokens using a frozen codebook. Unlike traditional quantization approaches, our method employs soft assignment and KL divergence guided quantization to address the unique challenges of graph data, which lacks natural tokenization structures. Our framework enables both LLM-based and traditional learning approaches, supporting true zero-shot transfer learning without requiring labeled data even in the source domain. Extensive experiments demonstrate state-of-the-art performance across multiple node classification benchmarks while maintaining compatibility with different LLM architectures, offering an elegant solution to bridging graph learning with LLMs.