Improved IR-based Bug Localization with Intelligent Relevance Feedback

TL;DR

This paper introduces BRaIn, a novel bug localization technique using Large Language Models to assess relevance and leverage feedback, significantly outperforming existing IR-based methods in accuracy and bug localization capability.

Contribution

BRaIn is the first approach to incorporate LLM-based relevance assessment and feedback for bug localization, bridging contextual gaps beyond traditional IR methods.

Findings

BRaIn outperforms baseline techniques by up to 89.5% in MAP and MRR.

It localizes 52% more bugs that baseline methods fail to identify.

Experimental results demonstrate significant improvements in bug localization accuracy.

Abstract

Software bugs pose a significant challenge during development and maintenance, and practitioners spend nearly 50% of their time dealing with bugs. Many existing techniques adopt Information Retrieval (IR) to localize a reported bug using textual and semantic relevance between bug reports and source code. However, they often struggle to bridge a critical gap between bug reports and code that requires in-depth contextual understanding, which goes beyond textual or semantic relevance. In this paper, we present a novel technique for bug localization - BRaIn - that addresses the contextual gaps by assessing the relevance between bug reports and code with Large Language Models (LLM). It then leverages the LLM's feedback (a.k.a., Intelligent Relevance Feedback) to reformulate queries and re-rank source documents, improving bug localization. We evaluate BRaIn using a benchmark dataset, Bench4BL, and three performance metrics and compare it against six baseline techniques from the literature. Our experimental results show that BRaIn outperforms baselines by 87.6%, 89.5%, and 48.8% margins in MAP, MRR, and HIT@K, respectively. Additionally, it can localize approximately 52% of bugs that cannot be localized by the baseline techniques due to the poor quality of corresponding bug reports. By addressing the contextual gaps and introducing Intelligent Relevance Feedback, BRaIn advances not only theory but also improves IR-based bug localization.