Fonte: Finding Bug Inducing Commits from Failures

TL;DR

Fonte is an efficient method for identifying bug inducing commits using only test coverage, significantly improving accuracy over existing techniques and reducing debugging costs in large-scale industrial projects.

Contribution

Fonte introduces a novel BIC identification approach combining fault localization with suspiciousness ranking based solely on test coverage, outperforming existing methods.

Findings

Fonte achieves at least 39% higher MRR than state-of-the-art techniques.

It can rank the actual BIC within top five commits for 87% of failures in industry-scale projects.

Fonte reduces BIC inspection costs by 32% on average.

Abstract

A Bug Inducing Commit (BIC) is a commit that introduces a software bug into the codebase. Knowing the relevant BIC for a given bug can provide valuable information for debugging as well as bug triaging. However, existing BIC identification techniques are either too expensive (because they require the failing tests to be executed against previous versions for bisection) or inapplicable at the debugging time (because they require post hoc artefacts such as bug reports or bug fixes). We propose Fonte, an efficient and accurate BIC identification technique that only requires test coverage. Fonte combines Fault Localisation (FL) with BIC identification and ranks commits based on the suspiciousness of the code elements that they modified. Fonte reduces the search space of BICs using failure coverage as well as a filter that detects commits that are merely style changes. Our empirical evaluation using 130 real-world BICs shows that Fonte significantly outperforms state-of-the-art BIC identification techniques based on Information Retrieval as well as neural code embedding models, achieving at least 39% higher MRR. We also report that the ranking scores produced by Fonte can be used to perform weighted bisection, further reducing the cost of BIC identification. Finally, we apply Fonte to a large-scale industry project with over 10M lines of code, and show that it can rank the actual BIC within the top five commits for 87% of the studied real batch-testing failures, and save the BIC inspection cost by 32% on average.