SBEST: Spectrum-Based Fault Localization Without Fault-Triggering Tests

TL;DR

SBEST is a novel fault localization method that leverages stack traces from crash reports combined with test coverage data to effectively locate faults without relying on fault-triggering tests, addressing a common practical challenge.

Contribution

This paper introduces SBEST, the first approach to integrate stack trace information with coverage data for fault localization without requiring fault-triggering tests.

Findings

SBEST improves MAP by 32.22% over baselines.

78.3% of buggy methods are reachable within 0.34 calls from stack traces.

Only 3.33% of bugs have fault-triggering tests available.

Abstract

Fault localization is a critical step in software maintenance. Yet, many existing techniques, such as Spectrum-Based Fault Localization (SBFL), rely heavily on the availability of fault-triggering tests to be effective. In practice, especially for crash-related bugs, such tests are frequently unavailable. Meanwhile, bug reports containing stack traces often serve as the only available evidence of runtime failures and provide valuable context for debugging. This study investigates the feasibility of using stack traces from crash reports as proxies for fault-triggering tests in SBFL. Our empirical analysis of 60 crash-report bugs in Defects4J reveals that only 3.33% of these bugs have fault-triggering tests available at the time of the bug report creation. However, 98.3% of bug fixes directly address the exception observed in the stack trace, and 78.3% of buggy methods are reachable within an average of 0.34 method calls from the stack trace. These findings underscore the diagnostic value of stack traces in the absence of failing tests. Motivated by these findings, we propose SBEST, a novel approach that integrates stack trace information with test coverage data to perform fault localization when fault-triggering tests are missing. SBEST shows an improvement, with a 32.22% increase in Mean Average Precision (MAP) and a 17.43% increase in Mean Reciprocal Rank (MRR) compared to baseline approaches under the scenario where fault-triggering tests are absent.