Does Programming Language Matter? An Empirical Study of Fuzzing Bug Detection

TL;DR

This study analyzes how programming language influences fuzzing bug detection effectiveness, revealing significant differences in bug characteristics and detection metrics across languages, which informs more tailored fuzzing strategies.

Contribution

It provides the first large-scale cross-language analysis of fuzzing bug detection, highlighting how language design impacts fuzzing effectiveness and bug characteristics.

Findings

C++ and Rust have higher bug detection frequencies.

Rust and Python expose more critical vulnerabilities despite low vulnerability ratios.

Unreproducible bugs are more common in Go and rare in Rust.

Abstract

Fuzzing has become a popular technique for automatically detecting vulnerabilities and bugs by generating unexpected inputs. In recent years, the fuzzing process has been integrated into continuous integration workflows (i.e., continuous fuzzing), enabling short and frequent testing cycles. Despite its widespread adoption, prior research has not examined whether the effectiveness of continuous fuzzing varies across programming languages. This study conducts a large-scale cross-language analysis to examine how fuzzing bug characteristics and detection efficiency differ among languages. We analyze 61,444 fuzzing bugs and 999,248 builds from 559 OSS-Fuzz projects categorized by primary language. Our findings reveal that (i) C++ and Rust exhibit higher fuzzing bug detection frequencies, (ii) Rust and Python show low vulnerability ratios but tend to expose more critical vulnerabilities, (iii) crash types vary across languages and unreproducible bugs are more frequent in Go but rare in Rust, and (iv) Python attains higher patch coverage but suffers from longer time-to-detection. These results demonstrate that fuzzing behavior and effectiveness are strongly shaped by language design, providing insights for language-aware fuzzing strategies and tool development.