LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification

TL;DR

This paper investigates how large language models enhance graph neural networks by analyzing their causal mechanisms using synthetic data and interchange interventions, leading to a new optimization module validated across datasets.

Contribution

It introduces a causal analysis framework for LLM-enhanced GNNs and proposes a plug-and-play module to improve information transfer.

Findings

Causal relationships in LLM-GNN interactions are elucidated.

The proposed optimization module improves GNN performance.

Experimental validation across multiple datasets confirms effectiveness.

Abstract

The use of large language models (LLMs) as feature enhancers to optimize node representations, which are then used as inputs for graph neural networks (GNNs), has shown significant potential in graph representation learning. However, the fundamental properties of this approach remain underexplored. To address this issue, we propose conducting a more in-depth analysis of this issue based on the interchange intervention method. First, we construct a synthetic graph dataset with controllable causal relationships, enabling precise manipulation of semantic relationships and causal modeling to provide data for analysis. Using this dataset, we conduct interchange interventions to examine the deeper properties of LLM enhancers and GNNs, uncovering their underlying logic and internal mechanisms. Building on the analytical results, we design a plug-and-play optimization module to improve the information transfer between LLM enhancers and GNNs. Experiments across multiple datasets and models validate the proposed module.