Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs

TL;DR

Plan-on-Graph (PoG) introduces a self-correcting, adaptive planning approach for knowledge graph-augmented large language models, improving reasoning accuracy and efficiency by decomposing questions, exploring reasoning paths, and self-correcting errors.

Contribution

This paper presents a novel self-correcting adaptive planning paradigm for KG-augmented LLMs, enabling dynamic exploration and correction of reasoning paths based on question semantics.

Findings

PoG outperforms baseline models on three real-world datasets.

PoG demonstrates improved reasoning accuracy and efficiency.

The mechanisms of Guidance, Memory, and Reflection are effective in adaptive planning.

Abstract

Large Language Models (LLMs) have shown remarkable reasoning capabilities on complex tasks, but they still suffer from out-of-date knowledge, hallucinations, and opaque decision-making. In contrast, Knowledge Graphs (KGs) can provide explicit and editable knowledge for LLMs to alleviate these issues. Existing paradigm of KG-augmented LLM manually predefines the breadth of exploration space and requires flawless navigation in KGs. However, this paradigm cannot adaptively explore reasoning paths in KGs based on the question semantics and self-correct erroneous reasoning paths, resulting in a bottleneck in efficiency and effect. To address these limitations, we propose a novel self-correcting adaptive planning paradigm for KG-augmented LLM named Plan-on-Graph (PoG), which first decomposes the question into several sub-objectives and then repeats the process of adaptively exploring reasoning paths, updating memory, and reflecting on the need to self-correct erroneous reasoning paths until arriving at the answer. Specifically, three important mechanisms of Guidance, Memory, and Reflection are designed to work together, to guarantee the adaptive breadth of self-correcting planning for graph reasoning. Finally, extensive experiments on three real-world datasets demonstrate the effectiveness and efficiency of PoG.