GR-Agent: Adaptive Graph Reasoning Agent under Incomplete Knowledge

TL;DR

This paper introduces GR-Agent, an adaptive reasoning agent for knowledge graph question answering that performs well even with incomplete KGs, by constructing an interactive environment and maintaining reasoning evidence.

Contribution

It proposes a novel methodology for benchmarking KGQA under incompleteness and introduces GR-Agent, a reasoning agent that outperforms baselines in such scenarios.

Findings

GR-Agent outperforms non-training baselines.

GR-Agent performs comparably to training-based methods.

Benchmark methodology effectively evaluates reasoning under KG incompleteness.

Abstract

Large language models (LLMs) achieve strong results on knowledge graph question answering (KGQA), but most benchmarks assume complete knowledge graphs (KGs) where direct supporting triples exist. This reduces evaluation to shallow retrieval and overlooks the reality of incomplete KGs, where many facts are missing and answers must be inferred from existing facts. We bridge this gap by proposing a methodology for constructing benchmarks under KG incompleteness, which removes direct supporting triples while ensuring that alternative reasoning paths required to infer the answer remain. Experiments on benchmarks constructed using our methodology show that existing methods suffer consistent performance degradation under incompleteness, highlighting their limited reasoning ability. To overcome this limitation, we present the Adaptive Graph Reasoning Agent (GR-Agent). It first constructs an interactive environment from the KG, and then formalizes KGQA as agent environment interaction within this environment. GR-Agent operates over an action space comprising graph reasoning tools and maintains a memory of potential supporting reasoning evidence, including relevant relations and reasoning paths. Extensive experiments demonstrate that GR-Agent outperforms non-training baselines and performs comparably to training-based methods under both complete and incomplete settings.