BigIssue: A Realistic Bug Localization Benchmark
Paul Kassianik, Erik Nijkamp, Bo Pang, Yingbo Zhou, Caiming Xiong

TL;DR
BigIssue is a new benchmark designed to evaluate bug localization techniques on realistic Java bugs, aiming to improve machine learning models' ability to identify bugs in real-world code repositories.
Contribution
The paper introduces BigIssue, a comprehensive benchmark dataset for realistic bug localization, encouraging development of models that consider full repository context.
Findings
Provides diverse real and synthetic Java bugs for evaluation
Highlights the importance of full repository context in bug localization
Aims to enhance bug localization and automatic program repair performance
Abstract
As machine learning tools progress, the inevitable question arises: How can machine learning help us write better code? With significant progress being achieved in natural language processing with models like GPT-3 and Bert, the applications of natural language processing techniques to code are starting to be explored. Most of the research has been focused on automatic program repair (APR), and while the results on synthetic or highly filtered datasets are promising, such models are hard to apply in real-world scenarios because of inadequate bug localization. We propose BigIssue: a benchmark for realistic bug localization. The goal of the benchmark is two-fold. We provide (1) a general benchmark with a diversity of real and synthetic Java bugs and (2) a motivation to improve bug localization capabilities of models through attention to the full repository context. With the introduction…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSoftware Engineering Research · Software Testing and Debugging Techniques · Software Reliability and Analysis Research
Methods{Dispute@FaQ-s}How to file a dispute with Expedia? · 15 Ways to Contact How can i speak to someone at Delta Airlines · Attention Is All You Need · Repair · Linear Layer · Cosine Annealing · Refunds@Expedia|||How do I get a full refund from Expedia? · Attention Dropout · Adam · Dense Connections
