Multi-hop Reasoning and Retrieval in Embedding Space: Leveraging Large Language Models with Knowledge

TL;DR

This paper introduces EMBRAG, an embedding-based retrieval reasoning framework that enhances large language models' ability to perform multi-hop reasoning with knowledge graphs, achieving state-of-the-art results in knowledge graph question answering.

Contribution

The paper presents a novel embedding-based reasoning framework that integrates knowledge graph retrieval with logical rule grounding to improve multi-hop reasoning in LLMs.

Findings

Achieves new state-of-the-art performance on KGQA datasets.

Effectively handles multi-hop reasoning with noisy and incomplete knowledge graphs.

Demonstrates robustness and accuracy improvements over existing methods.

Abstract

As large language models (LLMs) continue to grow in size, their abilities to tackle complex tasks have significantly improved. However, issues such as hallucination and the lack of up-to-date knowledge largely remain unresolved. Knowledge graphs (KGs), which serve as symbolic representations of real-world knowledge, offer a reliable source for enhancing reasoning. Integrating KG retrieval into LLMs can therefore strengthen their reasoning by providing dependable knowledge. Nevertheless, due to limited understanding of the underlying knowledge graph, LLMs may struggle with queries that have multiple interpretations. Additionally, the incompleteness and noise within knowledge graphs may result in retrieval failures. To address these challenges, we propose an embedding-based retrieval reasoning framework EMBRAG. In this approach, the model first generates multiple logical rules grounded in knowledge graphs based on the input query. These rules are then applied to reasoning in the embedding space, guided by the knowledge graph, ensuring more robust and accurate reasoning. A reranker model further interprets these rules and refines the results. Extensive experiments on two benchmark KGQA datasets demonstrate that our approach achieves the new state-of-the-art performance in KG reasoning tasks.