Runtime Execution Traces Guided Automated Program Repair with Multi-Agent Debate

TL;DR

TraceRepair is a multi-agent framework that uses runtime execution traces as shared constraints to improve automated program repair, significantly outperforming existing LLM-based methods on benchmark datasets.

Contribution

It introduces a novel multi-agent approach leveraging runtime facts as shared constraints for more accurate and dynamic program repair.

Findings

Correctly fixed 392 defects on Defects4J benchmark

Outperformed existing LLM-based approaches significantly

Demonstrated improved efficiency and generalization on new bug datasets

Abstract

Automated Program Repair (APR) struggles with complex logic errors and silent failures. Current LLM-based APR methods are mostly static, relying on source code and basic test outputs, which fail to accurately capture complex runtime behaviors and dynamic data dependencies. While incorporating runtime evidence like execution traces exposes concrete state transitions, a single LLM interpreting this in isolation often overfits to specific hypotheses, producing patches that satisfy tests by coincidence rather than correct logic. Therefore, runtime evidence should act as objective constraints rather than mere additional input. We propose TraceRepair, a multi-agent framework that leverages runtime facts as shared constraints for patch validation. A probe agent captures execution snapshots of critical variables to form an objective repair basis. Meanwhile, a committee of specialized agents cross-verifies candidate patches to expose inconsistencies and iteratively refine them. Evaluated on the Defects4J benchmark, TraceRepair correctly fixes 392 defects, substantially outperforming existing LLM-based approaches. Extensive experiments demonstrate improved efficiency and strong generalization on a newly constructed dataset of recent bugs, confirming that performance gains arise from dynamic reasoning rather than memorization.