Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach

TL;DR

This paper introduces a novel approach combining Large Language Models and Graph Neural Networks to effectively learn representations from heterogeneous graphs without prior type information or extensive preprocessing.

Contribution

We propose a generalized method that leverages LLMs to process diverse graph data formats and types, enabling GNNs to operate effectively on heterogeneous graphs without prior knowledge.

Findings

Method achieves comparable or superior performance on downstream tasks.

No need for prior node/edge type labels or extensive preprocessing.

Theoretical analysis confirms the method's effectiveness.

Abstract

Graph representation learning methods are highly effective in handling complex non-Euclidean data by capturing intricate relationships and features within graph structures. However, traditional methods face challenges when dealing with heterogeneous graphs that contain various types of nodes and edges due to the diverse sources and complex nature of the data. Existing Heterogeneous Graph Neural Networks (HGNNs) have shown promising results but require prior knowledge of node and edge types and unified node feature formats, which limits their applicability. Recent advancements in graph representation learning using Large Language Models (LLMs) offer new solutions by integrating LLMs' data processing capabilities, enabling the alignment of various graph representations. Nevertheless, these methods often overlook heterogeneous graph data and require extensive preprocessing. To address these limitations, we propose a novel method that leverages the strengths of both LLM and GNN, allowing for the processing of graph data with any format and type of nodes and edges without the need for type information or special preprocessing. Our method employs LLM to automatically summarize and classify different data formats and types, aligns node features, and uses a specialized GNN for targeted learning, thus obtaining effective graph representations for downstream tasks. Theoretical analysis and experimental validation have demonstrated the effectiveness of our method.