MASTER: A Multi-Agent System with LLM Specialized MCTS

TL;DR

MASTER introduces a multi-agent system leveraging specialized LLM-based MCTS to improve strategic planning and efficiency in complex tasks, achieving state-of-the-art results in question answering benchmarks.

Contribution

The paper proposes a novel multi-agent framework with specialized LLM-based MCTS that dynamically adjusts agent recruitment and communication, addressing sampling inefficiencies in reward estimation.

Findings

Achieves 76% accuracy on HotpotQA

Achieves 80% accuracy on WebShop

Sets new state-of-the-art performance

Abstract

Large Language Models (LLM) are increasingly being explored for problem-solving tasks. However, their strategic planning capability is often viewed with skepticism. Recent studies have incorporated the Monte Carlo Tree Search (MCTS) algorithm to augment the planning capacity of LLM. Despite its potential, MCTS relies on extensive sampling simulations to approximate the true reward distribution, which leads to two primary issues. Firstly, MCTS is effective for tasks like the Game of Go, where simulation results can yield objective rewards (e.g., 1 for a win and 0 for a loss). However, for tasks such as question answering, the result of a simulation is the answer to the question, which cannot yield an objective reward without the ground truth. Secondly, obtaining statistically significant reward estimations typically requires a sample size exceeding 30 simulations, resulting in excessive token usage and time consumption. To address these challenges, we present the Multi-Agent System with Tactical Execution and Reasoning using LLM Specialized MCTS (MASTER), a novel framework that coordinates agent recruitment and communication through LLM specialized MCTS. This system autonomously adjusts the number of agents based on task complexity and ensures focused communication among them. Comprehensive experiments across various tasks demonstrate the effectiveness of our proposed framework. It achieves 76% accuracy on HotpotQA and 80% on WebShop, setting new state-of-the-art performance on these datasets.