RouterKGQA: Specialized--General Model Routing for Constraint-Aware Knowledge Graph Question Answering

TL;DR

RouterKGQA is a hybrid framework that combines specialized and general models for knowledge graph question answering, improving accuracy and efficiency by collaborative reasoning and constraint-aware filtering.

Contribution

It introduces a novel specialized--general model collaboration framework with constraint-aware filtering and efficient workflows for KGQA.

Findings

Outperforms previous best by 3.57 F1 points and 0.49 Hits@1.

Requires only 1.15 LLM calls per question on average.

Achieves better performance with lower inference cost.

Abstract

Knowledge graph question answering (KGQA) is a promising approach for mitigating LLM hallucination by grounding reasoning in structured and verifiable knowledge graphs. Existing approaches fall into two paradigms: retrieval-based methods utilize small specialized models, which are efficient but often produce unreachable paths and miss implicit constraints, while agent-based methods utilize large general models, which achieve stronger structural grounding at substantially higher cost. We propose RouterKGQA, a framework for specialized--general model collaboration, in which a specialized model generates reasoning paths and a general model performs KG-guided repair only when needed, improving performance at minimal cost. We further equip the specialized with constraint-aware answer filtering, which reduces redundant answers. In addition, we design a more efficient general agent workflow, further lowering inference cost. Experimental results show that RouterKGQA outperforms the previous best by 3.57 points in F1 and 0.49 points in Hits@1 on average across benchmarks, while requiring only 1.15 average LLM calls per question. Codes and models are available at https://github.com/Oldcircle/RouterKGQA.