Large Language Models are Good Relational Learners

TL;DR

This paper introduces Rel-LLM, a novel approach that combines graph neural networks with large language models to better understand and reason over complex relational data, outperforming existing methods.

Contribution

Rel-LLM is the first to integrate GNN-based structured prompts with LLMs for relational deep learning, preserving relational structures and improving reasoning capabilities.

Findings

Rel-LLM outperforms existing methods on key RDL tasks.

The GNN encoder effectively captures entity relationships and temporal dependencies.

Structured prompts enable LLMs to process complex relational data more efficiently.

Abstract

Large language models (LLMs) have demonstrated remarkable capabilities across various domains, yet their application to relational deep learning (RDL) remains underexplored. Existing approaches adapt LLMs by traversing relational links between entities in a database and converting the structured data into flat text documents. Still, this text-based serialization disregards critical relational structures, introduces redundancy, and often exceeds standard LLM context lengths. We introduce Rel-LLM, a novel architecture that utilizes a graph neural network (GNN)- based encoder to generate structured relational prompts for LLMs within a retrieval-augmented generation (RAG) framework. Unlike traditional text-based serialization approaches, our method preserves the inherent relational structure of databases while enabling LLMs to effectively process and reason over complex entity relationships. Specifically, the GNN encoder extracts a local subgraph around an entity to build feature representations that contain relevant entity relationships and temporal dependencies. These representations are transformed into structured prompts using a denormalization process, effectively allowing the LLM to reason over relational structures. Through extensive experiments, we demonstrate that Rel-LLM outperforms existing methods on key RDL tasks, offering a scalable and efficient approach to integrating LLMs with structured data sources. Code is available at https://github.com/smiles724/Rel-LLM.