NOCL: Node-Oriented Conceptualization LLM for Graph Tasks without Message Passing

TL;DR

NOCL introduces a novel language-based framework for graph tasks that converts node attributes into natural language and semantic embeddings, enabling effective zero-shot learning and broad applicability without message passing.

Contribution

The paper presents NOCL, a new LLM-based approach that extends to non-textual graphs and reduces token lengths, improving zero-shot generalization and unifying graph tasks in a language format.

Findings

Competitive supervised performance with MPNNs

Superior zero-shot generalization

Significant token length reduction (up to 93.9%)

Abstract

Graphs are essential for modeling complex interactions across domains such as social networks, biology, and recommendation systems. Traditional Graph Neural Networks, particularly Message Passing Neural Networks (MPNNs), rely heavily on supervised learning, limiting their generalization and applicability in label-scarce scenarios. Recent self-supervised approaches still require labeled fine-tuning, limiting their effectiveness in zero-shot scenarios. Meanwhile, Large Language Models (LLMs) excel in natural language tasks but face significant challenges when applied to graphs, including preserving reasoning abilities, managing extensive token lengths from rich node attributes, and being limited to textual-attributed graphs (TAGs) and a single level task. To overcome these limitations, we propose the Node-Oriented Conceptualization LLM (NOCL), a novel framework that leverages two core techniques: 1) node description, which converts heterogeneous node attributes into structured natural language, extending LLM from TAGs to non-TAGs; 2) node concept, which encodes node descriptions into compact semantic embeddings using pretrained language models, significantly reducing token lengths by up to 93.9% compared to directly using node descriptions. Additionally, our NOCL employs graph representation descriptors to unify graph tasks at various levels into a shared, language-based query format, paving a new direction for Graph Foundation Models. Experimental results validate NOCL's competitive supervised performance relative to traditional MPNNs and hybrid LLM-MPNN methods and demonstrate superior generalization in zero-shot settings.