Finding the Needle in the Crash Stack: Industrial-Scale Crash Root Cause Localization with AutoCrashFL

TL;DR

AutoCrashFL is an LLM-based crash localization tool that effectively identifies root causes in large industrial software using only crashdumps and source code, outperforming baseline methods.

Contribution

This paper introduces AutoCrashFL, a novel LLM-powered crash localization approach that operates without coverage profiling, suitable for industrial-scale software.

Findings

AutoCrashFL identified 30% of crashes at the top rank.

AutoCrashFL outperformed baseline by 13 percentage points.

AutoCrashFL is effective for complex bugs and provides confidence estimates.

Abstract

Fault Localization (FL) aims to identify root causes of program failures. FL typically targets failures observed from test executions, and as such, often involves dynamic analyses to improve accuracy, such as coverage profiling or mutation testing. However, for large industrial software, measuring coverage for every execution is prohibitively expensive, making the use of such techniques difficult. To address these issues and apply FL in an industrial setting, this paper proposes AutoCrashFL, an LLM agent for the localization of crashes that only requires the crashdump from the Program Under Test (PUT) and access to the repository of the corresponding source code. We evaluate AutoCrashFL against real-world crashes of SAP HANA, an industrial software project consisting of more than 35 million lines of code. Experiments reveal that AutoCrashFL is more effective in localization, as it identified 30% crashes at the top, compared to 17% achieved by the baseline. Through thorough analysis, we find that AutoCrashFL has attractive practical properties: it is relatively more effective for complex bugs, and it can indicate confidence in its results. Overall, these results show the practicality of LLM agent deployment on an industrial scale.