A Large-Scale Collection Of (Non-)Actionable Static Code Analysis Reports

TL;DR

This paper introduces NASCAR, a large-scale dataset of over 1 million Java static code analysis warnings, to help distinguish actionable issues from non-actionable ones and improve developer alert management.

Contribution

The paper presents a novel methodology for collecting and categorizing SCA warnings, creating a large, publicly available dataset to advance research in filtering and understanding static analysis alerts.

Findings

Generated a dataset of over 1 million Java warnings

Provided tools for categorizing actionable vs. non-actionable warnings

Facilitated research to reduce alert fatigue in static analysis

Abstract

Static Code Analysis (SCA) tools, while invaluable for identifying potential coding problems, functional bugs, or vulnerabilities, often generate an overwhelming number of warnings, many of which are non-actionable. This overload of alerts leads to ``alert fatigue'', a phenomenon where developers become desensitized to warnings, potentially overlooking critical issues and ultimately hindering productivity and code quality. Analyzing these warnings and training machine learning models to identify and filter them requires substantial datasets, which are currently scarce, particularly for Java. This scarcity impedes efforts to improve the accuracy and usability of SCA tools and mitigate the effects of alert fatigue. In this paper, we address this gap by introducing a novel methodology for collecting and categorizing SCA warnings, effectively distinguishing actionable from non-actionable ones. We further leverage this methodology to generate a large-scale dataset of over 1 million entries of Java source code warnings, named NASCAR: (Non-)Actionable Static Code Analysis Reports. To facilitate follow-up research in this domain, we make both the dataset and the tools used to generate it publicly available.