Deep Semantic Graph Learning via LLM based Node Enhancement

TL;DR

This paper introduces a novel graph learning framework that integrates Large Language Model-generated semantic node features with Graph Transformer architecture, significantly enhancing node classification performance by capturing deep textual semantics and structural information.

Contribution

It presents a new method combining LLM-based semantic node features with Graph Transformer, improving deep semantic understanding in graph learning tasks.

Findings

LLM-enhanced node features improve classification accuracy

The model captures both local and global graph structures

Experimental results outperform existing methods

Abstract

Graph learning has attracted significant attention due to its widespread real-world applications. Current mainstream approaches rely on text node features and obtain initial node embeddings through shallow embedding learning using GNNs, which shows limitations in capturing deep textual semantics. Recent advances in Large Language Models (LLMs) have demonstrated superior capabilities in understanding text semantics, transforming traditional text feature processing. This paper proposes a novel framework that combines Graph Transformer architecture with LLM-enhanced node features. Specifically, we leverage LLMs to generate rich semantic representations of text nodes, which are then processed by a multi-head self-attention mechanism in the Graph Transformer to capture both local and global graph structural information. Our model utilizes the Transformer's attention mechanism to dynamically aggregate neighborhood information while preserving the semantic richness provided by LLM embeddings. Experimental results demonstrate that the LLM-enhanced node features significantly improve the performance of graph learning models on node classification tasks. This approach shows promising results across multiple graph learning tasks, offering a practical direction for combining graph networks with language models.