DeepAnalyze: Learning to Localize Crashes at Scale

TL;DR

DeepAnalyze is a scalable, data-driven approach that uses machine learning to accurately localize crash causes in large-scale error reporting systems, reducing manual rule creation and maintenance.

Contribution

We introduce DeepAnalyze, a novel multi-task sequence labeling model that effectively localizes crash causes across diverse applications with minimal additional training.

Findings

Accurately localizes crashes in large-scale real-world data

Generalizes to new applications with little to no retraining

Outperforms rule-based crash localization methods

Abstract

Crash localization, an important step in debugging crashes, is challenging when dealing with an extremely large number of diverse applications and platforms and underlying root causes. Large-scale error reporting systems, e.g., Windows Error Reporting (WER), commonly rely on manually developed rules and heuristics to localize blamed frames causing the crashes. As new applications and features are routinely introduced and existing applications are run under new environments, developing new rules and maintaining existing ones become extremely challenging. We propose a data-driven solution to address the problem. We start with the first large-scale empirical study of 362K crashes and their blamed methods reported to WER by tens of thousands of applications running in the field. The analysis provides valuable insights on where and how the crashes happen and what methods to blame for the crashes. These insights enable us to develop DeepAnalyze, a novel multi-task sequence labeling approach for identifying blamed frames in stack traces. We evaluate our model with over a million real-world crashes from four popular Microsoft applications and show that DeepAnalyze, trained with crashes from one set of applications, not only accurately localizes crashes of the same applications, but also bootstraps crash localization for other applications with zero to very little additional training data.