PathFinder: MCTS and LLM Feedback-based Path Selection for Multi-Hop Question Answering

TL;DR

PathFinder enhances multi-hop question answering by integrating Monte Carlo Tree Search with LLM feedback to generate and filter reasoning paths, leading to improved accuracy on benchmark datasets.

Contribution

It introduces a novel method combining MCTS and LLM verification to generate high-quality training paths for multi-hop QA, reducing errors and hallucinations.

Findings

Improves multi-hop QA accuracy on benchmarks

Filters erroneous reasoning paths effectively

Enhances training data quality for reasoning models

Abstract

Multi-hop question answering is a challenging task in which language models must reason over multiple steps to reach the correct answer. With the help of Large Language Models and their reasoning capabilities, existing systems are able to think and decompose an input question over multiple steps to analyze, retrieve, and reason. However, training-based approaches for this problem still suffer from LLM hallucinations and incorrect reasoning paths that hinder performance. Hence, we propose PATHFINDER, an approach that: (i) uses Monte Carlo Tree Search to generate training path traces, (ii) improves training data quality by filtering erroneous and lengthy traces using sub-answer recall and LLM-as-a-judge verification, and (iii) reformulates sub-queries to handle failed retrieval cases. By following these steps, we demonstrate that PATHFINDER improves the performance of multi-hop QA over public benchmark datasets.