BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering

TL;DR

BYOKG-RAG is a novel framework that combines large language models with specialized graph retrieval tools to improve question answering over knowledge graphs, especially for custom KGs, by iteratively refining graph context and answers.

Contribution

It introduces a multi-strategy graph retrieval framework that enhances KGQA robustness and generalization, addressing limitations of existing LLM-based approaches.

Findings

Outperforms previous graph retrieval methods by 4.5% points on five benchmarks.

Demonstrates better generalization to custom knowledge graphs.

Effectively combines LLMs with graph tools for iterative refinement.

Abstract

Knowledge graph question answering (KGQA) presents significant challenges due to the structural and semantic variations across input graphs. Existing works rely on Large Language Model (LLM) agents for graph traversal and retrieval; an approach that is sensitive to traversal initialization, as it is prone to entity linking errors and may not generalize well to custom ("bring-your-own") KGs. We introduce BYOKG-RAG, a framework that enhances KGQA by synergistically combining LLMs with specialized graph retrieval tools. In BYOKG-RAG, LLMs generate critical graph artifacts (question entities, candidate answers, reasoning paths, and OpenCypher queries), and graph tools link these artifacts to the KG and retrieve relevant graph context. The retrieved context enables the LLM to iteratively refine its graph linking and retrieval, before final answer generation. By retrieving context from different graph tools, BYOKG-RAG offers a more general and robust solution for QA over custom KGs. Through experiments on five benchmarks spanning diverse KG types, we demonstrate that BYOKG-RAG outperforms the second-best graph retrieval method by 4.5% points while showing better generalization to custom KGs. BYOKG-RAG framework is open-sourced at https://github.com/awslabs/graphrag-toolkit.