Collaborative Agents for Automated Program Repair in Ruby

TL;DR

This paper introduces RAMP, a lightweight, multi-agent framework for automated program repair in Ruby that outperforms prior methods by efficiently generating and refining fixes through feedback-driven iterations.

Contribution

RAMP is a novel multi-agent, feedback-driven framework for Ruby APR that avoids large databases and costly fine-tuning, demonstrating improved repair performance.

Findings

RAMP achieves 67% pass@1 on Ruby in XCodeEval.

RAMP converges within five iterations.

Test generation and self-reflection are key to RAMP's success.

Abstract

Automated Program Repair (APR) has advanced rapidly with Large Language Models (LLMs), but most existing methods remain computationally expensive, and focused on a small set of languages. Ruby, despite its widespread use in web development and the persistent challenges faced by its developers, has received little attention in APR research. In this paper, we introduce RAMP, a novel lightweight framework that formulates program repair as a feedback-driven, iterative process for Ruby. RAMP employs a team of collaborative agents that generate targeted tests, reflect on errors, and refine candidate fixes until a correct solution is found. Unlike prior approaches, RAMP is designed to avoid reliance on large multilingual repair databases or costly fine-tuning, instead operating directly on Ruby through lightweight prompting and test-driven feedback. Evaluation on the XCodeEval benchmark shows that RAMP achieves a pass@1 of 67% on Ruby, outper-forming prior approaches. RAMP converges quickly within five iterations, and ablation studies confirm that test generation and self-reflection are key drivers of its performance. Further analysis shows that RAMP is particularly effective at repairing wrong answers, compilation errors, and runtime errors. Our approach provides new insights into multi-agent repair strategies, and establishes a foundation for extending LLM-based debugging tools to under-studied languages.