Industry-scale IR-based Bug Localization: A Perspective from Facebook

TL;DR

This paper evaluates IR-based bug localization at Facebook's industrial scale, introduces Bug2Commit to address unique challenges, and demonstrates its superior accuracy over existing methods in real-world applications.

Contribution

The paper presents Bug2Commit, a novel IR-based bug localization tool tailored for industrial-scale, complex bug reports, and evaluates its effectiveness against existing techniques.

Findings

Bug2Commit outperforms existing IR methods by up to 17% in accuracy.

Effective in diverse Facebook applications like crash analysis and performance regressions.

Reduces bug triaging time and improves bug attribution in large-scale industrial settings.

Abstract

We explore the application of Information Retrieval (IR) based bug localization methods at a large industrial setting, Facebook. Facebook's code base evolves rapidly, with thousands of code changes being committed to a monolithic repository every day. When a bug is detected, it is often time-sensitive and imperative to identify the commit causing the bug in order to either revert it or fix it. This is complicated by the fact that bugs often manifest with complex and unwieldy features, such as stack traces and other metadata. Code commits also have various features associated with them, ranging from developer comments to test results. This poses unique challenges to bug localization methods, making it a highly non-trivial operation. In this paper we lay out several practical concerns for industry-level IR-based bug localization, and propose Bug2Commit, a tool that is designed to address these concerns. We also assess the effectiveness of existing IR-based localization techniques from the software engineering community, and find that in the presence of complex queries or documents, which are common at Facebook, existing approaches do not perform as well as Bug2Commit. We evaluate Bug2Commit on three applications at Facebook: client-side crashes from the mobile app, server-side performance regressions, and mobile simulation tests for performance. We find that Bug2Commit outperforms the accuracy of existing approaches by up to 17%, leading to reduced time for triaging regressions and attributing bugs found in simulations.