From Query to Logic: Ontology-Driven Multi-Hop Reasoning in LLMs

TL;DR

This paper introduces ORACLE, a training-free framework that enhances multi-hop question answering in LLMs by integrating knowledge graphs and logical reasoning, leading to more accurate and interpretable results.

Contribution

The paper presents ORACLE, a novel approach combining LLMs with knowledge graph-based logical reasoning for improved multi-hop question answering without additional training.

Findings

Achieves competitive performance on standard MQA benchmarks.

Produces more logical and interpretable reasoning chains.

Rivals state-of-the-art models like DeepSeek-R1.

Abstract

Large Language Models (LLMs), despite their success in question answering, exhibit limitations in complex multi-hop question answering (MQA) tasks that necessitate non-linear, structured reasoning. This limitation stems from their inability to adequately capture deep conceptual relationships between entities. To overcome this challenge, we present **ORACLE** (**O**ntology-driven **R**easoning **A**nd **C**hain for **L**ogical **E**ucidation), a training-free framework that combines LLMs' generative capabilities with the structural benefits of knowledge graphs. Our approach operates through three stages: (1) dynamic construction of question-specific knowledge ontologies using LLMs, (2) transformation of these ontologies into First-Order Logic reasoning chains, and (3) systematic decomposition of the original query into logically coherent sub-questions. Experimental results on several standard MQA benchmarks show that our framework achieves highly competitive performance, rivaling current state-of-the-art models like DeepSeek-R1. Detailed analyses further confirm the effectiveness of each component, while demonstrating that our method generates more logical and interpretable reasoning chains than existing approaches.