CodeScout: An Effective Recipe for Reinforcement Learning of Code Search Agents

TL;DR

CodeScout demonstrates that a reinforcement learning approach using only a standard Unix terminal can effectively train code search agents, outperforming larger models on multiple benchmarks.

Contribution

The paper introduces a reinforcement learning recipe for code search agents that requires no specialized tools, achieving strong results with standard environments.

Findings

Outperforms larger models on three benchmarks

Achieves near performance of closed models like Claude Sonnet

Uses standard Unix terminal without complex tools

Abstract

A prerequisite for coding agents to perform tasks on large repositories is code localization - the identification of relevant files, classes, and functions to work on. While repository-level code localization has been performed using embedding-based retrieval approaches such as vector search, recent work has focused on developing agents to localize relevant code either as a standalone precursor to or interleaved with performing actual work. Most prior methods on agentic code search equip the agent with complex, specialized tools, such as repository graphs derived from static analysis. In this paper, we demonstrate that, with an effective reinforcement learning recipe, a coding agent equipped with nothing more than a standard Unix terminal can be trained to achieve strong results. Our experiments on three benchmarks (SWE-Bench Verified, Pro, and Lite) reveal that our models consistently achieve superior or competitive performance over 2-18x larger base and post-trained LLMs and sometimes approach performance provided by closed models like Claude Sonnet, even when using specialized scaffolds. Our work particularly focuses on techniques for re-purposing existing coding agent environments for code search, reward design, and RL optimization. We release the resulting model family, CodeScout, along with all our code and data for the community to build upon.