GitBug-Java: A Reproducible Benchmark of Recent Java Bugs
Andr\'e Silva, Nuno Saavedra, Martin Monperrus
TL;DR
GitBug-Java is a new, reproducible benchmark of 199 recent Java bugs from open-source repositories, designed to support evaluation of automatic program repair and fault localization methods.
Contribution
It introduces a reproducible, up-to-date bug benchmark based on recent Java bugs, addressing limitations of existing datasets like Defects4J.
Findings
Provides a fully-reproducible environment for bug fixes
Includes 199 recent Java bugs from 55 repositories
Enhances evaluation of APR and FL techniques
Abstract
Bug-fix benchmarks are essential for evaluating methodologies in automatic program repair (APR) and fault localization (FL). However, existing benchmarks, exemplified by Defects4J, need to evolve to incorporate recent bug-fixes aligned with contemporary development practices. Moreover, reproducibility, a key scientific principle, has been lacking in bug-fix benchmarks. To address these gaps, we present GitBug-Java, a reproducible benchmark of recent Java bugs. GitBug-Java features 199 bugs extracted from the 2023 commit history of 55 notable open-source repositories. The methodology for building GitBug-Java ensures the preservation of bug-fixes in fully-reproducible environments. We publish GitBug-Java at https://github.com/gitbugactions/gitbug-java.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Malware Detection Techniques · Software Testing and Debugging Techniques · Software Engineering Research