SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement

TL;DR

SWE-Search introduces a multi-agent framework combining Monte Carlo Tree Search with self-improvement mechanisms to enhance software agents' performance in complex tasks, enabling iterative refinement and collaborative decision-making.

Contribution

The paper presents SWE-Search, a novel multi-agent system integrating MCTS with LLM-based evaluation and self-feedback for improved software engineering tasks.

Findings

23% performance improvement on SWE-bench benchmark

Performance scales with increased inference-time compute

Effective collaborative decision-making among agents

Abstract

Software engineers operating in complex and dynamic environments must continuously adapt to evolving requirements, learn iteratively from experience, and reconsider their approaches based on new insights. However, current large language model (LLM)-based software agents often follow linear, sequential processes that prevent backtracking and exploration of alternative solutions, limiting their ability to rethink their strategies when initial approaches prove ineffective. To address these challenges, we propose SWE-Search, a multi-agent framework that integrates Monte Carlo Tree Search (MCTS) with a self-improvement mechanism to enhance software agents' performance on repository-level software tasks. SWE-Search extends traditional MCTS by incorporating a hybrid value function that leverages LLMs for both numerical value estimation and qualitative evaluation. This enables self-feedback loops where agents iteratively refine their strategies based on both quantitative numerical evaluations and qualitative natural language assessments of pursued trajectories. The framework includes a SWE-Agent for adaptive exploration, a Value Agent for iterative feedback, and a Discriminator Agent that facilitates multi-agent debate for collaborative decision-making. Applied to the SWE-bench benchmark, our approach demonstrates a 23% relative improvement in performance across five models compared to standard open-source agents without MCTS. Our analysis reveals how performance scales with increased inference-time compute through deeper search, providing a pathway to improve software agents without requiring larger models or additional training data. This highlights the potential of self-evaluation driven search techniques in complex software engineering environments.