Many Tools, Few Exploitable Vulnerabilities: A Survey of 246 Static Code Analyzers for Security

TL;DR

This survey reviews 246 static code analyzers for security, revealing that most focus on limited weaknesses, detect rarely exploitable vulnerabilities, and are evaluated with small, non-standard benchmarks.

Contribution

It provides the first comprehensive overview of static security analyzers across multiple dimensions, highlighting gaps and limitations in current evaluation practices.

Findings

Most analyzers target limited weaknesses

Detected vulnerabilities are rarely exploitable

Evaluations use small, custom benchmarks

Abstract

Static security analysis is a widely used technique for detecting software vulnerabilities across a wide range of weaknesses, application domains, and programming languages. While prior work surveyed static analyzes for specific weaknesses or application domains, no overview of the entire security landscape exists. We present a systematic literature review of 246 static security analyzers concerning their targeted vulnerabilities, application domains, analysis techniques, evaluation methods, and limitations. We observe that most analyzers focus on a limited set of weaknesses, that the vulnerabilities they detect are rarely exploitable, and that evaluations use custom benchmarks that are too small to enable robust assessment.