Reasoning in Action: MCTS-Driven Knowledge Retrieval for Large Language Models

TL;DR

This paper presents a novel reasoning-aware knowledge retrieval method for large language models that combines retrieval and reasoning, using a Monte Carlo Tree Search-inspired approach to improve dialogue responses.

Contribution

It introduces a coarse-to-fine knowledge retrieval framework that aligns with logical conversation structures and employs MCTS-inspired search for better knowledge navigation.

Findings

Enhanced relevance of retrieved knowledge to reasoning processes

Increased diversity and informativeness of generated responses

Improved alignment with human conversation logic

Abstract

Large language models (LLMs) typically enhance their performance through either the retrieval of semantically similar information or the improvement of their reasoning capabilities. However, a significant challenge remains in effectively integrating both retrieval and reasoning strategies to optimize LLM performance. In this paper, we introduce a reasoning-aware knowledge retrieval method that enriches LLMs with information aligned to the logical structure of conversations, moving beyond surface-level semantic similarity. We follow a coarse-to-fine approach for knowledge retrieval. First, we identify a contextually relevant sub-region of the knowledge base, ensuring that all sentences within it are relevant to the context topic. Next, we refine our search within this sub-region to extract knowledge that is specifically relevant to the reasoning process. Throughout both phases, we employ the Monte Carlo Tree Search-inspired search method to effectively navigate through knowledge sentences using common keywords. Experiments on two multi-turn dialogue datasets demonstrate that our knowledge retrieval approach not only aligns more closely with the underlying reasoning in human conversations but also significantly enhances the diversity of the retrieved knowledge, resulting in more informative and creative responses.