# Large-Scale Evaluation of Method-Level Bug Localization with FinerBench4BL

## TL;DR

This paper introduces FinerBench4BL, a large-scale evaluation framework for method-level bug localization, revealing that method-level techniques are less accurate but more efficient than file-level methods.

## Contribution

The paper presents FinerBench4BL, a novel framework for evaluating method-level bug localization techniques, including dataset transformation and tailored evaluation methods.

## Key findings

- Method-level techniques decrease accuracy compared to file-level techniques.
- Method-level techniques improve debugging efficiency.
- FinerBench4BL enables large-scale, systematic evaluation of bug localization methods.

## Abstract

Bug localization is an important aspect of software maintenance because it can locate modules that need to be changed to fix a specific bug. Although method-level bug localization is helpful for developers, there are only a few tools and techniques for this task; moreover, there is no large-scale framework for their evaluation. In this paper, we present FinerBench4BL, an evaluation framework for method-level information retrieval-based bug localization techniques, and a comparative study using this framework. This framework was semi-automatically constructed from Bench4BL, a file-level bug localization evaluation framework, using a repository transformation approach. We converted the original file-level version repositories provided by Bench4BL into method-level repositories by repository transformation. Method-level data components such as oracle methods can also be automatically derived by applying the oracle generation approach via bug-commit linking in Bench4BL to the generated method repositories. Furthermore, we tailored existing file-level bug localization technique implementations at the method level. We created a framework for method-level evaluation by merging the generated dataset and implementations. The comparison results show that the method-level techniques decreased accuracy whereas improved debugging efficiency compared to file-level techniques.