EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering

TL;DR

EPERM is a three-stage framework that enhances reasoning in knowledge graph question answering by retrieving, filtering, and weighting evidence paths to improve LLM performance.

Contribution

This paper introduces EPERM, a novel method that reformulates KGQA as a graphical model and emphasizes the importance of evidence paths for reasoning.

Findings

EPERM outperforms existing KGQA methods on benchmark datasets.

Filtering and weighting evidence paths improves reasoning accuracy.

EPERM effectively reduces hallucinations in LLM-based KGQA.

Abstract

Due to the remarkable reasoning ability, Large language models (LLMs) have demonstrated impressive performance in knowledge graph question answering (KGQA) tasks, which find answers to natural language questions over knowledge graphs (KGs). To alleviate the hallucinations and lack of knowledge issues of LLMs, existing methods often retrieve the question-related information from KGs to enrich the input context. However, most methods focus on retrieving the relevant information while ignoring the importance of different types of knowledge in reasoning, which degrades their performance. To this end, this paper reformulates the KGQA problem as a graphical model and proposes a three-stage framework named the Evidence Path Enhanced Reasoning Model (EPERM) for KGQA. In the first stage, EPERM uses the fine-tuned LLM to retrieve a subgraph related to the question from the original knowledge graph. In the second stage, EPERM filters out the evidence paths that faithfully support the reasoning of the questions, and score their importance in reasoning. Finally, EPERM uses the weighted evidence paths to reason the final answer. Since considering the importance of different structural information in KGs for reasoning, EPERM can improve the reasoning ability of LLMs in KGQA tasks. Extensive experiments on benchmark datasets demonstrate that EPERM achieves superior performances in KGQA tasks.