Duplicate Bug Report Detection: How Far Are We?
Ting Zhang, DongGyun Han, Venkatesh Vinayakarao, Ivana Clairine Irsan, Bowen Xu, Ferdian Thung, David Lo, Lingxiao Jiang

TL;DR
This paper compares various duplicate bug report detection techniques using a new benchmark, revealing that simpler methods often outperform recent sophisticated approaches in practical settings.
Contribution
It introduces a new benchmark for fair comparison of DBRD techniques and provides insights into their real-world effectiveness.
Findings
A simple technique outperforms recent sophisticated methods on most projects.
Data age and issue tracking system choice significantly affect accuracy.
Practitioners' simple techniques can achieve results comparable to research tools.
Abstract
Many Duplicate Bug Report Detection (DBRD) techniques have been proposed in the research literature. The industry uses some other techniques. Unfortunately, there is insufficient comparison among them, and it is unclear how far we have been. This work fills this gap by comparing the aforementioned techniques. To compare them, we first need a benchmark that can estimate how a tool would perform if applied in a realistic setting today. Thus, we first investigated potential biases that affect the fair comparison of the accuracy of DBRD techniques. Our experiments suggest that data age and issue tracking system choice cause a significant difference. Based on these findings, we prepared a new benchmark. We then used it to evaluate DBRD techniques to estimate better how far we have been. Surprisingly, a simpler technique outperforms recently proposed sophisticated techniques on most projects…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
