Injecting Knowledge Graphs into Large Language Models

TL;DR

This paper presents a novel, resource-efficient method for integrating Knowledge Graphs into Large Language Models using Knowledge Graph Embeddings, enhancing reasoning capabilities without high computational costs.

Contribution

It extends graph embedding techniques to the KG domain, enabling model-agnostic, efficient, graph-aware reasoning in LLMs.

Findings

Improves reasoning performance over baselines

Achieves better accuracy-efficiency trade-off

Compatible with any LLMs

Abstract

Integrating structured knowledge from Knowledge Graphs (KGs) into Large Language Models (LLMs) remains a key challenge for symbolic reasoning. Existing methods mainly rely on prompt engineering or fine-tuning, which lose structural fidelity or incur high computational costs. Building on recent encoding techniques which integrate graph embeddings within the LLM input as tokens, we extend this paradigm to the KG domain by leveraging Knowledge Graph Embedding (KGE) models, thus enabling graph-aware reasoning. Our approach is model-agnostic, resource-efficient, and compatible with any LLMs. Extensive experimentation on synthetic and real-world datasets shows that our method improves reasoning performance over established baselines, further achieving the best trade-off in terms of accuracy and efficiency against state-of-the-art LLMs.