Efficient Multi-Hop Question Answering over Knowledge Graphs via LLM Planning and Embedding-Guided Search

TL;DR

This paper introduces efficient, verifiable multi-hop question answering methods over knowledge graphs using hybrid algorithms that combine LLM planning, embedding-guided search, and knowledge distillation, significantly improving speed and grounding accuracy.

Contribution

It proposes two novel hybrid algorithms for multi-hop QA that enhance efficiency and verifiability, including a single-call LLM planning approach and an embedding-guided neural search, with model compression for practical deployment.

Findings

LLM-Guided Planning achieves micro-F1 > 0.90 with a single LLM call.

Embedding-Guided Neural Search speeds up over 100 times with competitive accuracy.

Grounded reasoning outperforms ungrounded generation on MetaQA.

Abstract

Multi-hop question answering over knowledge graphs remains computationally challenging due to the combinatorial explosion of possible reasoning paths. Recent approaches rely on expensive Large Language Model (LLM) inference for both entity linking and path ranking, limiting their practical deployment. Additionally, LLM-generated answers often lack verifiable grounding in structured knowledge. We present two complementary hybrid algorithms that address both efficiency and verifiability: (1) LLM-Guided Planning that uses a single LLM call to predict relation sequences executed via breadth-first search, achieving near-perfect accuracy (micro-F1 > 0.90) while ensuring all answers are grounded in the knowledge graph, and (2) Embedding-Guided Neural Search that eliminates LLM calls entirely by fusing text and graph embeddings through a lightweight 6.7M-parameter edge scorer, achieving over 100 times speedup with competitive accuracy. Through knowledge distillation, we compress planning capability into a 4B-parameter model that matches large-model performance at zero API cost. Evaluation on MetaQA demonstrates that grounded reasoning consistently outperforms ungrounded generation, with structured planning proving more transferable than direct answer generation. Our results show that verifiable multi-hop reasoning does not require massive models at inference time, but rather the right architectural inductive biases combining symbolic structure with learned representations.