Attention Please: Consider Mockito when Evaluating Newly Proposed Automated Program Repair Techniques
Shangwen Wang, Ming Wen, Xiaoguang Mao, Deheng Yang

TL;DR
This paper investigates the importance of including Mockito bugs in the evaluation of automated program repair techniques, revealing that current tools perform poorly on these bugs and suggesting the need for broader evaluation benchmarks.
Contribution
It highlights the necessity of considering Mockito bugs in APR evaluations and proposes external package addition as a new repair action to improve techniques.
Findings
Mockito bugs are not more complex than other bugs
State-of-the-art tools perform poorly on Mockito bugs
External package addition is a key repair action for Mockito bugs
Abstract
Automated program repair (APR) has attracted widespread attention in recent years with substantial techniques being proposed. Meanwhile, a number of benchmarks have been established for evaluating the performances of APR techniques, among which Defects4J is one of the most wildly used benchmark. However, bugs in Mockito, a project augmented in a later-version of Defects4J, do not receive much attention by recent researches. In this paper, we aim at investigating the necessity of considering Mockito bugs when evaluating APR techniques. Our findings show that: 1) Mockito bugs are not more complex for repairing compared with bugs from other projects; 2) the bugs repaired by the state-of-the-art tools share the same repair patterns compared with those patterns required to repair Mockito bugs; however, 3) the state-of-the-art tools perform poorly on Mockito bugs (Nopol can only correctly fix…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Testing and Debugging Techniques · Software Reliability and Analysis Research · Software Engineering Research
