The benefits of query-based KGQA systems for complex and temporal questions in LLM era

TL;DR

This paper presents a multi-stage query-based knowledge graph question-answering framework that improves handling of complex multi-hop and temporal questions in the era of large language models, demonstrating robustness and novel entity linking techniques.

Contribution

It introduces a novel multi-stage query-based KGQA approach with enhanced entity linking and predicate matching, improving performance on complex temporal and multi-hop questions.

Findings

Enhanced performance on multi-hop benchmarks

Improved temporal question answering accuracy

Robustness across diverse datasets

Abstract

Large language models excel in question-answering (QA) yet still struggle with multi-hop reasoning and temporal questions. Query-based knowledge graph QA (KGQA) offers a modular alternative by generating executable queries instead of direct answers. We explore multi-stage query-based framework for WikiData QA, proposing multi-stage approach that enhances performance on challenging multi-hop and temporal benchmarks. Through generalization and rejection studies, we evaluate robustness across multi-hop and temporal QA datasets. Additionally, we introduce a novel entity linking and predicate matching method using CoT reasoning. Our results demonstrate the potential of query-based multi-stage KGQA framework for improving multi-hop and temporal QA with small language models. Code and data: https://github.com/ar2max/NLDB-KGQA-System