E-APR: Mapping the Effectiveness of Automated Program Repair

TL;DR

E-APR is a novel framework that visualizes and analyzes the difficulty of buggy programs for automated repair techniques, helping evaluate and improve their effectiveness.

Contribution

The paper introduces E-APR, a new method for explaining and evaluating the performance of APR techniques through instance space visualization.

Findings

E-APR reveals areas of hard and easy buggy programs.

It analyzes the diversity and quality of buggy datasets.

It identifies strengths and weaknesses of existing APR methods.

Abstract

Automated Program Repair (APR) is a fast growing area with numerous new techniques being developed to tackle one of the most challenging software engineering problems. APR techniques have shown promising results, giving us hope that one day it will be possible for software to repair itself. In this paper, we focus on the problem of objective performance evaluation of APR techniques. We introduce a new approach, Explaining Automated Program Repair (E-APR), which identifies features of buggy programs that explain why a particular instance is difficult for an APR technique. E-APR is used to examine the diversity and quality of the buggy programs used by most researchers, and analyse the strengths and weaknesses of existing APR techniques. E-APR visualises an instance space of buggy programs, with each buggy program represented as a point in the space. The instance space is constructed to reveal areas of hard and easy buggy programs, and enables the strengths and weaknesses of APR techniques to be identified.