Beyond Chains: Bridging Large Language Models and Knowledge Bases in Complex Question Answering

TL;DR

This paper introduces PDRR, a four-stage framework that enhances complex question answering by combining semantic parsing, knowledge retrieval, and reasoning with large language models, outperforming existing methods.

Contribution

The paper presents PDRR, a novel four-stage approach that integrates question decomposition, knowledge retrieval, and LLM reasoning to handle complex KBQA tasks more effectively.

Findings

PDRR outperforms existing methods on various LLM backbones.

Effective handling of both chain-structured and non-chain complex questions.

Improved accuracy and reasoning capabilities in KBQA tasks.

Abstract

Knowledge Base Question Answering (KBQA) aims to answer natural language questions using structured knowledge from KBs. While LLM-only approaches offer generalization, they suffer from outdated knowledge, hallucinations, and lack of transparency. Chain-based KG-RAG methods address these issues by incorporating external KBs, but are limited to simple chain-structured questions due to the absence of planning and logical structuring. Inspired by semantic parsing methods, we propose PDRR: a four-stage framework consisting of Predict, Decompose, Retrieve, and Reason. Our method first predicts the question type and decomposes the question into structured triples. Then retrieves relevant information from KBs and guides the LLM as an agent to reason over and complete the decomposed triples. Experimental results demonstrate that PDRR consistently outperforms existing methods across various LLM backbones and achieves superior performance on both chain-structured and non-chain complex questions.