Enhancing Large Language Models with Reward-guided Tree Search for Knowledge Graph Question and Answering

TL;DR

This paper introduces RTSoG, a novel framework that enhances KGQA by decomposing questions, guiding reasoning path retrieval with reward-based tree search, and stacking paths for improved answer accuracy, outperforming existing methods.

Contribution

The paper proposes a training-free, reward-guided tree search framework for KGQA that effectively exploits both new and historical reasoning paths, addressing complex question semantics.

Findings

Achieves 8.7% improvement on GrailQA dataset.

Achieves 7.0% improvement on WebQSP dataset.

Demonstrates effectiveness across four datasets.

Abstract

Recently, large language models (LLMs) have demonstrated impressive performance in Knowledge Graph Question Answering (KGQA) tasks, which aim to find answers based on knowledge graphs (KGs) for natural language questions. Existing LLMs-based KGQA methods typically follow the Graph Retrieval-Augmented Generation (GraphRAG) paradigm, which first retrieves reasoning paths from the large KGs, and then generates the answers based on them. However, these methods emphasize the exploration of new optimal reasoning paths in KGs while ignoring the exploitation of historical reasoning paths, which may lead to sub-optimal reasoning paths. Additionally, the complex semantics contained in questions may lead to the retrieval of inaccurate reasoning paths. To address these issues, this paper proposes a novel and training-free framework for KGQA tasks called Reward-guided Tree Search on Graph (RTSoG). RTSoG decomposes an original question into a series of simpler and well-defined sub-questions to handle the complex semantics. Then, a Self-Critic Monte Carlo Tree Search (SC-MCTS) guided by a reward model is introduced to iteratively retrieve weighted reasoning paths as contextual knowledge. Finally, it stacks the weighted reasoning paths according to their weights to generate the final answers. Extensive experiments on four datasets demonstrate the effectiveness of RTSoG. Notably, it achieves 8.7\% and 7.0\% performance improvement over the state-of-the-art method on the GrailQA and the WebQSP respectively.