A Comparative Study of Fuzzers and Static Analysis Tools for Finding Memory Unsafety in C and C++

TL;DR

This study empirically compares fuzzers and static analyzers for detecting memory safety issues in C/C++ programs, revealing their complementary strengths and limitations.

Contribution

It provides a comprehensive evaluation of 5 static analyzers and 13 fuzzers on over 100 vulnerabilities, highlighting their detection patterns and practical implications.

Findings

Fuzzers find a similar set of bugs, while static analyzers report more diverse vulnerabilities.

The union of all fuzzers and static analyzers is nearly disjoint, indicating complementarity.

Static analyzers and fuzzers have different detection capabilities and limitations.

Abstract

Over 70% of security vulnerabilities in critical software systems today result from memory safety violations. To address this challenge, fuzzing and static analysis are widely used automated methods to discover such vulnerabilities. Fuzzing generates random program inputs to identify faults at runtime, while static analysis reasons about the code to detect potential vulnerabilities. Although these techniques share a common goal, they take fundamentally different approaches and have evolved largely independently. In this paper, we present an empirical analysis of five static analyzers and 13 fuzzers, applied to over 100 known security vulnerabilities in C/C++ programs. We measure the detection rate for each tool and vulnerability to evaluate how the approaches differ and complement each other. We find that fuzzers discover a very similar set of bugs, while static analyzers report more diverse sets, and identify clear leaders for each group. Comparing the union of all fuzzers with that of all static analyzers, we observe they are nearly disjoint. In a second step, we manually validate the report-to-bug mapping we developed for the evaluation and discuss more qualitative aspects of limitations, usability, and integration into the development process. We examine how widely these bug finding tools are used in critical open-source projects. We advise developers on choosing tools to harden their software and identify barriers to adoption as well as future research opportunities.