Broccoli: Bug localization with the help of text search engines

TL;DR

This paper explores how normal text search engines can enhance bug localization in software development, demonstrating improved performance over existing methods through empirical evaluation on open source projects.

Contribution

It introduces Broccoli, a search engine-based bug localization approach, and reveals a flaw in current benchmarking strategies, proposing a more accurate evaluation method.

Findings

Search engines improve bug localization accuracy

Broccoli outperforms seven state-of-the-art algorithms

Considering repository state at bug report time increases evaluation accuracy

Abstract

Bug localization is a tedious activity in the bug fixing process in which a software developer tries to locate bugs in the source code described in a bug report. Since this process is time-consuming and requires additional knowledge about the software project, information retrieval techniques can aid the bug localization process. In this paper, we investigate if normal text search engines can improve existing bug localization approaches. In a case study, we evaluate the performance of our search engine approach Broccoli against seven state-of-the-art bug localization algorithms on 82 open source projects in two data sets. Our results show that including a search engine can increase the performance of the bug localization and that it is a useful extension to existing approaches. As part of our analysis we also exposed a flaw in a commonly used benchmark strategy, i.e., that files of a single release are considered. To increase the number of detectable files, we mitigate this flaw by considering the state of the software repository at the time of the bug report. Our results show that using single releases may lead to an underestimation of the the prediction performance.