Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering

TL;DR

This paper introduces Ontology-Guided Reverse Thinking (ORT), a novel framework that enhances large language models' ability to perform multi-hop knowledge graph question answering by constructing reasoning paths from purposes to conditions.

Contribution

The paper proposes a new reverse reasoning framework guided by ontology, improving LLMs' performance on KGQA tasks beyond existing entity matching methods.

Findings

Achieves state-of-the-art results on WebQSP and CWQ datasets.

Significantly improves multi-hop reasoning in KGQA.

Demonstrates the effectiveness of reverse thinking in LLMs.

Abstract

Large language models (LLMs) have shown remarkable capabilities in natural language processing. However, in knowledge graph question answering tasks (KGQA), there remains the issue of answering questions that require multi-hop reasoning. Existing methods rely on entity vector matching, but the purpose of the question is abstract and difficult to match with specific entities. As a result, it is difficult to establish reasoning paths to the purpose, which leads to information loss and redundancy. To address this issue, inspired by human reverse thinking, we propose Ontology-Guided Reverse Thinking (ORT), a novel framework that constructs reasoning paths from purposes back to conditions. ORT operates in three key phases: (1) using LLM to extract purpose labels and condition labels, (2) constructing label reasoning paths based on the KG ontology, and (3) using the label reasoning paths to guide knowledge retrieval. Experiments on the WebQSP and CWQ datasets show that ORT achieves state-of-the-art performance and significantly enhances the capability of LLMs for KGQA.