"False negative -- that one is going to kill you": Understanding Industry Perspectives of Static Analysis based Security Testing

TL;DR

This study explores developers' perceptions and challenges with static analysis security testing tools through interviews, revealing gaps and guiding future improvements in SAST design and adoption.

Contribution

It provides qualitative insights into developer expectations, beliefs, and challenges with SAST tools, highlighting gaps in current practices and suggesting future research directions.

Findings

Developers often underestimate SAST limitations.

Misalignment between developer expectations and SAST capabilities.

Identified gaps in SAST tool usability and effectiveness.

Abstract

The demand for automated security analysis techniques, such as static analysis based security testing (SAST) tools continues to increase. To develop SASTs that are effectively leveraged by developers for finding vulnerabilities, researchers and tool designers must understand how developers perceive, select, and use SASTs, what they expect from the tools, whether they know of the limitations of the tools, and how they address those limitations. This paper describes a qualitative study that explores the assumptions, expectations, beliefs, and challenges experienced by developers who use SASTs. We perform in-depth, semi-structured interviews with 20 practitioners who possess a diverse range of software development expertise, as well as a variety of unique security, product, and organizational backgrounds. We identify $17$ key findings that shed light on developer perceptions and desires related to SASTs, and also expose gaps in the status quo - challenging long-held beliefs in SAST design priorities. Finally, we provide concrete future directions for researchers and practitioners rooted in an analysis of our findings.