SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning

TL;DR

This paper introduces Repository-Centric Learning (RCL), a new paradigm for training small language models to specialize in specific code repositories, leading to improved performance and efficiency over traditional task-centric methods.

Contribution

The paper proposes RCL as a paradigm shift, designing a repository-focused training approach that enhances small language models' ability to understand and generalize within specific software environments.

Findings

SWE-Spot-4B models outperform larger models on multiple SWE tasks.

RCL improves training sample efficiency and reduces inference costs.

Repository mastery complements general coding capabilities.

Abstract

The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs). However, they suffer from a fundamental capability gap: unlike frontier large models, they lack the inference-time strong generalization to work with complicated, unfamiliar codebases. We identify that the prevailing Task-Centric Learning (TCL) paradigm, which scales exposure across disparate repositories, fails to address this limitation. In response, we propose Repository-Centric Learning (RCL), a paradigm shift that prioritizes vertical repository depth over horizontal task breadth, suggesting SLMs must internalize the "physics" of a target software environment through parametric knowledge acquisition, rather than attempting to recover it via costly inference-time search. Following this new paradigm, we design a four-unit Repository-Centric Experience, transforming static codebases into interactive learning signals, to train SWE-Spot-4B, a family of highly compact models built as repo-specialized experts that breaks established scaling trends, outperforming open-weight models up to larger (e.g., CWM by Meta, Qwen3-Coder-30B) and surpassing/matching efficiency-focused commercial models (e.g., GPT-4.1-mini, GPT-5-nano) across multiple SWE tasks. Further analysis reveals that RCL yields higher training sample efficiency and lower inference costs, emphasizing that for building efficient intelligence, repository mastery is a distinct and necessary dimension that complements general coding capability.