Towards Better Static Code Analysis Reports: Sentence Transformer-based Filtering of Non-Actionable Alerts

TL;DR

This paper introduces STAF, a transformer-based filtering method that significantly reduces non-actionable static analysis alerts, improving developer focus and tool usability.

Contribution

The paper presents a novel Sentence Transformer-based approach for classifying static analysis findings, achieving higher accuracy than existing filtering methods.

Findings

Achieved an F1 score of 89% in classifying actionable alerts.

Reduced non-actionable findings by at least 11% within projects.

Improved filtering performance by at least 6% across projects.

Abstract

Static code analysis (SCA) tools are widely used as effective ways to detect bugs and vulnerabilities in software systems. However, the reports generated by these tools often contain a large number of non-actionable findings, which can overwhelm developers to the point of ignoring them altogether -- this phenomenon is known as "alert fatigue". In this paper, we combat alert fatigue by proposing STAF: Sentence Transformer-based Actionability Filtering. Our approach leverages a transformer based architecture with sentence embeddings to classify findings into actionable and non-actionable categories. Evaluating STAF on a large dataset of reports from Java projects, we demonstrate that our method can effectively reduce the number of non-actionable findings while maintaining a high level of accuracy in identifying actionable issues. The results show that our approach can improve the usability of static analysis tools reaching an F1 score of 89%, outperforming existing methods for SCA warning filtering by at least 11% in a within-project setting and by at least 6% in a cross-project setting. By providing a more focused and relevant set of findings, we aim to enhance the overall effectiveness of static analysis in software development.