Fuzzing at Scale: The Untold Story of the Scheduler

TL;DR

This paper demonstrates that dynamic scheduling strategies significantly improve bug detection in large-scale fuzzing, matching the impact of advanced fuzzers, by intelligently allocating fuzzing time across numerous programs.

Contribution

It introduces and evaluates dynamic schedulers for large-scale fuzzing, showing they outperform naive approaches and are crucial for effective bug discovery at scale.

Findings

Dynamic schedulers find more bugs than naive ones.

Performance of schedulers can rival state-of-the-art fuzzers.

Fuzzing 5,000 programs yielded 4,908 bugs.

Abstract

How to search for bugs in 1,000 programs using a pre-existing fuzzer and a standard PC? We consider this problem and show that a well-designed strategy that determines which programs to fuzz and for how long can greatly impact the number of bugs found across the programs. In fact, the impact of employing an effective strategy is comparable to that of utilizing a state-of-the-art fuzzer. The considered problem is referred to as fuzzing at scale, and the strategy as scheduler. We show that besides a naive scheduler, that allocates equal fuzz time to all programs, we can consider dynamic schedulers that adjust time allocation based on the ongoing fuzzing progress of individual programs. Such schedulers are superior because they lead both to higher number of total found bugs and to higher number of found bugs for most programs. The performance gap between naive and dynamic schedulers can be as wide (or even wider) as the gap between two fuzzers. Our findings thus suggest that the problem of advancing schedulers is fundamental for fuzzing at scale. We develop several schedulers and leverage the most sophisticated one to fuzz simultaneously our newly compiled benchmark of around 5,000 Ubuntu programs, and detect 4908 bugs.