Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models

TL;DR

Co-PatcheR introduces a collaborative system of small, specialized models for software patching, achieving higher success rates with less resource use than large monolithic models by dividing tasks into localization, generation, and validation.

Contribution

This work presents the first collaborative patching system with component-specific small models, improving efficiency and effectiveness over large models in software patching tasks.

Findings

Achieves 46% resolved rate on SWE-bench-Verified with only 3x14B models

Outperforms state-of-the-art methods using fewer and smaller models

Validated through extensive ablation studies on training data, model size, and scaling strategies

Abstract

Motivated by the success of general-purpose large language models (LLMs) in software patching, recent works started to train specialized patching models. Most works trained one model to handle the end-to-end patching pipeline (including issue localization, patch generation, and patch validation). However, it is hard for a small model to handle all tasks, as different sub-tasks have different workflows and require different expertise. As such, by using a 70 billion model, SOTA methods can only reach up to 41% resolved rate on SWE-bench-Verified. Motivated by the collaborative nature, we propose Co-PatcheR, the first collaborative patching system with small and specialized reasoning models for individual components. Our key technique novelties are the specific task designs and training recipes. First, we train a model for localization and patch generation. Our localization pinpoints the suspicious lines through a two-step procedure, and our generation combines patch generation and critique. We then propose a hybrid patch validation that includes two models for crafting issue-reproducing test cases with and without assertions and judging patch correctness, followed by a majority vote-based patch selection. Through extensive evaluation, we show that Co-PatcheR achieves 46% resolved rate on SWE-bench-Verified with only 3 x 14B models. This makes Co-PatcheR the best patcher with specialized models, requiring the least training resources and the smallest models. We conduct a comprehensive ablation study to validate our recipes, as well as our choice of training data number, model size, and testing-phase scaling strategy.