Mining Bug Repositories for Multi-Fault Programs

TL;DR

This paper extends bug datasets like Defects4J and BugsInPy to include multi-fault entries, enabling more realistic evaluation of debugging tools by simulating real-world multi-bug scenarios.

Contribution

It introduces a method to identify and locate multiple bugs within single software entries, enhancing existing datasets for more comprehensive testing.

Findings

Created datasets with multiple bugs per entry

Maintained original dataset properties and usability

Enabled realistic multi-fault software evaluation

Abstract

Datasets such as Defects4J and BugsInPy that contain bugs from real-world software projects are necessary for a realistic evaluation of automated debugging tools. However these datasets largely identify only a single bug in each entry, while real-world software projects (including those used in Defects4J and BugsInPy) typically contain multiple bugs at the same time. We lift this limitation and describe an extension to these datasets in which multiple bugs are identified in individual entries. We use test case transplantation and fault location translation, in order to expose and locate the bugs, respectively. We thus provide datasets of true multi-fault versions within real-world software projects, which maintain the properties and usability of the original datasets.