How Different Is It Between Machine-Generated and Developer-Provided Patches? An Empirical Study on The Correct Patches Generated by Automated Program Repair Techniques
Shangwen Wang, Ming Wen, Liqian Chen, Xin Yi, Xiaoguang, Mao

TL;DR
This empirical study investigates the differences between machine-generated and developer-provided patches in automated program repair, revealing significant syntactic variations and distribution patterns across projects and techniques.
Contribution
The paper provides a comprehensive analysis of correct patches from state-of-the-art APR techniques, offering insights and guidelines for future patch correctness assessment.
Findings
25.4% of correct patches differ syntactically from developer patches
Distribution of patches varies across projects and techniques
Patches for larger bugs tend to differ more from developer patches
Abstract
Background: Over the years, Automated Program Repair (APR) has attracted much attention from both academia and industry since it can reduce the costs in fixing bugs. However, how to assess the patch correctness remains to be an open challenge. Two widely adopted ways to approach this challenge, including manually checking and validating using automated generated tests, are biased (i.e., suffering from subjectivity and low precision respectively). Aim: To address this concern, we propose to conduct an empirical study towards understanding the correct patches that are generated by existing state-of-the-art APR techniques, aiming at providing guidelines for future assessment of patches. Method: To this end, we first present a Literature Review (LR) on the reported correct patches generated by recent techniques on the Defects4J benchmark and collect 177 correct patches after a process of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Software Reliability and Analysis Research · Software Engineering Research
