Program Repair with Minimal Edits Using CodeT5

TL;DR

This paper introduces a method using fine-tuned CodeT5 to suggest minimal edits for program repair, achieving high accuracy and close similarity to correct solutions, thus aiding debugging and understanding.

Contribution

It presents a novel approach of fine-tuning CodeT5 for minimal edit program repair, demonstrating superior performance over baseline models.

Findings

CodeT5 achieves pass@100 of 91.95%

Average edit distance of 6.84 for suggested programs

Effective in repairing introductory programming problems

Abstract

Programmers often struggle to identify and fix bugs in their programs. In recent years, many language models (LMs) have been proposed to fix erroneous programs and support error recovery. However, the LMs tend to generate solutions that differ from the original input programs. This leads to potential comprehension difficulties for users. In this paper, we propose an approach to suggest a correct program with minimal repair edits using CodeT5. We fine-tune a pre-trained CodeT5 on code pairs of wrong and correct programs and evaluate its performance with several baseline models. The experimental results show that the fine-tuned CodeT5 achieves a pass@100 of 91.95% and an average edit distance of the most similar correct program of 6.84, which indicates that at least one correct program can be suggested by generating 100 candidate programs. We demonstrate the effectiveness of LMs in suggesting program repair with minimal edits for solving introductory programming problems.