Empowering RepoQA-Agent based on Reinforcement Learning Driven by Monte-carlo Tree Search

TL;DR

This paper introduces RepoSearch-R1, a reinforcement learning framework driven by Monte-carlo Tree Search, enabling efficient, data-compliant repository question-answering without model distillation or external supervision.

Contribution

The paper presents a novel agentic reinforcement learning approach using MCTS that improves repository QA performance and training efficiency without relying on costly distillation or external data.

Findings

16.0% improvement over no-retrieval methods

19.5% improvement over iterative retrieval methods

33% increase in training efficiency

Abstract

Repository-level software engineering tasks require large language models (LLMs) to efficiently navigate and extract information from complex codebases through multi-turn tool interactions. Existing approaches face significant limitations: training-free, in-context learning methods struggle to guide agents effectively in tool utilization and decision-making based on environmental feedback, while training-based approaches typically rely on costly distillation from larger LLMs, introducing data compliance concerns in enterprise environments. To address these challenges, we introduce RepoSearch-R1, a novel agentic reinforcement learning framework driven by Monte-carlo Tree Search (MCTS). This approach allows agents to generate diverse, high-quality reasoning trajectories via self-training without requiring model distillation or external supervision. Based on RepoSearch-R1, we construct a RepoQA-Agent specifically designed for repository question-answering tasks. Comprehensive evaluation on repository question-answering tasks demonstrates that RepoSearch-R1 achieves substantial improvements of answer completeness: 16.0% enhancement over no-retrieval methods, 19.5% improvement over iterative retrieval methods, and 33% increase in training efficiency compared to general agentic reinforcement learning approaches. Our cold-start training methodology eliminates data compliance concerns while maintaining robust exploration diversity and answer completeness across repository-level reasoning tasks.