Semantic Refinement with LLMs for Graph Representations

TL;DR

This paper introduces GES, a graph representation learning framework that uses in-graph exemplars and LLMs to adaptively refine node semantics, improving performance across diverse graph types.

Contribution

It presents a data-centric semantic refinement method leveraging in-graph exemplars and LLMs, addressing structure-semantics heterogeneity in graph learning.

Findings

Consistent improvements on semantics-rich graphs.

Effective semantic refinement in structure-dominated graphs.

Outperforms existing methods in diverse graph domains.

Abstract

Graph-structured data exhibit substantial heterogeneity in where their predictive signals originate: in some domains, node-level semantics dominate, while in others, structural patterns play a central role. This structure-semantics heterogeneity implies that no graph learning model with a fixed inductive bias can generalize optimally across diverse graph domains. However, most existing methods address this challenge from the model side by incrementally injecting new inductive biases, which remains fundamentally limited given the open-ended diversity of real-world graphs. In this work, we take a data-centric perspective and treat node semantics as a task-adaptive variable. We propose a Graph-Exemplar-guided Semantic Refinement (GES) framework for graph representation learning which -- unlike existing LLM-enhanced methods that generate node descriptions without graph context -- leverages structurally and semantically similar nodes from the graph itself to guide semantic refinement. Specifically, a GNN is first trained to produce predictive states, which along with structural and semantic similarity are used to retrieve in-graph exemplars that inform an LLM in refining node descriptions. We evaluate our approach on both text-rich and text-free graphs. Results show consistent improvements on semantics-rich and structure-dominated graphs, demonstrating the effectiveness of data-centric semantic refinement under structure-semantics heterogeneity.