LLM-based Discriminative Reasoning for Knowledge Graph Question Answering

TL;DR

This paper introduces READS, a discriminative reasoning approach for knowledge graph question answering that reduces hallucinations in LLMs, leading to improved accuracy and state-of-the-art results on key benchmarks.

Contribution

The paper proposes a novel discriminative subtask reformulation and inference strategy for LLM-based KGQA, addressing hallucination issues and enhancing performance.

Findings

Outperforms multiple strong comparison methods

Achieves state-of-the-art results on WebQSP and CWQ benchmarks

Reduces hallucination and ungrounded reasoning in LLMs

Abstract

Large language models (LLMs) based on generative pre-trained Transformer have achieved remarkable performance on knowledge graph question-answering (KGQA) tasks. However, LLMs often produce ungrounded subgraph planning or reasoning results in KGQA due to the hallucinatory behavior brought by the generative paradigm. To tackle this issue, we propose READS to reformulate the KGQA process into discriminative subtasks, which simplifies the search space for each subtasks. Based on the subtasks, we design a new corresponding discriminative inference strategy to conduct the reasoning for KGQA, thereby alleviating hallucination and ungrounded reasoning issues in LLMs. Experimental results show that the proposed approach outperforms multiple strong comparison methods, along with achieving state-of-the-art performance on widely used benchmarks WebQSP and CWQ.