Zero-Shot Multi-Hop Question Answering via Monte-Carlo Tree Search with Large Language Models

TL;DR

This paper presents a zero-shot multi-hop question answering framework using Monte-Carlo tree search with large language models, reducing error propagation and improving efficiency without domain-specific examples.

Contribution

It introduces a novel zero-shot prompting method combined with Monte-Carlo tree search for better reasoning path identification in multi-hop QA tasks.

Findings

Outperforms existing methods on HotpotQA, 2WikiMultihopQA, MuSiQue

Achieves over 10-fold increase in reasoning speed with minimal performance loss

Effectively mitigates error propagation in multi-hop reasoning

Abstract

Recent advances in large language models (LLMs) have significantly impacted the domain of multi-hop question answering (MHQA), where systems are required to aggregate information and infer answers from disparate pieces of text. However, the autoregressive nature of LLMs inherently poses a challenge as errors may accumulate if mistakes are made in the intermediate reasoning steps. This paper introduces Monte-Carlo tree search for Zero-shot multi-hop Question Answering (MZQA), a framework based on Monte-Carlo tree search (MCTS) to identify optimal reasoning paths in MHQA tasks, mitigating the error propagation from sequential reasoning processes. Unlike previous works, we propose a zero-shot prompting method, which relies solely on instructions without the support of hand-crafted few-shot examples that typically require domain expertise. We also introduce a behavioral cloning approach (MZQA-BC) trained on self-generated MCTS inference trajectories, achieving an over 10-fold increase in reasoning speed with bare compromise in performance. The efficacy of our method is validated on standard benchmarks such as HotpotQA, 2WikiMultihopQA, and MuSiQue, demonstrating that it outperforms existing frameworks.