The Right Prompts for the Job: Repair Code-Review Defects with Large Language Model

TL;DR

This paper explores how to effectively use Large Language Models with specially designed prompts to repair code review defects, achieving a repair rate of nearly 73%, thus improving automatic program repair methods.

Contribution

It introduces optimized prompting strategies for LLMs to enhance code review defect repair, demonstrating significant improvements over existing approaches.

Findings

Achieved a repair rate of 72.97% with the best prompt.

Compared various prompts across multiple LLMs and datasets.

Showed improved effectiveness and practicality of APR techniques.

Abstract

Automatic program repair (APR) techniques have the potential to reduce manual efforts in uncovering and repairing program defects during the code review (CR) process. However, the limited accuracy and considerable time costs associated with existing APR approaches hinder their adoption in industrial practice. One key factor is the under-utilization of review comments, which provide valuable insights into defects and potential fixes. Recent advancements in Large Language Models (LLMs) have enhanced their ability to comprehend natural and programming languages, enabling them to generate patches based on review comments. This paper conducts a comprehensive investigation into the effective utilization of LLMs for repairing CR defects. In this study, various prompts are designed and compared across mainstream LLMs using two distinct datasets from human reviewers and automated checkers. Experimental results demonstrate a remarkable repair rate of 72.97% with the best prompt, highlighting a substantial improvement in the effectiveness and practicality of automatic repair techniques.