Monte Carlo Tree Search for Execution-Guided Program Repair with Large Language Models

TL;DR

This paper introduces CodePilot, a hybrid framework combining Monte Carlo Tree Search with large language models to improve execution-guided program repair for real-world software issues, demonstrating significant performance gains.

Contribution

The paper presents a novel hybrid approach integrating MCTS with large language models for execution-guided program repair, addressing long-horizon reasoning challenges.

Findings

CodePilot achieves a 24.67% issue resolution rate on SWE-bench Lite.

Combining symbolic search with neural models enhances repair effectiveness.

Hierarchical fault localization improves search efficiency.

Abstract

Automated program repair with large language models remains challenging at the repository level due to long-horizon reasoning requirements and the limitations of autoregressive decoding. We present CodePilot, a hybrid framework that integrates Monte Carlo Tree Search (MCTS) with large language models to enable execution-guided program repair for real-world GitHub issues. CodePilot performs hierarchical fault localization from repository to file and function level, explores diverse patch trajectories using MCTS, and leverages execution feedback as a reward signal to guide search and refinement. The framework further incorporates confidence-calibrated generation to selectively refine low-confidence outputs. Experiments on SWE-bench Lite demonstrate that CodePilot achieves a 24.67% issue resolution rate using open-weight models, outperforming comparable baselines. These results suggest that combining symbolic search with neural language models is an effective strategy for scalable, execution-aware software engineering automation.