Is Stateful Fuzzing Really Challenging?

TL;DR

This paper discusses the challenges in developing and benchmarking effective fuzzers for stateful systems, contrasting them with the more established stateless fuzzing tools like AFL.

Contribution

It analyzes the inherent difficulties in creating and evaluating stateful fuzzers, highlighting the gap in research and benchmarking methods.

Findings

Stateful fuzzers are harder to develop and benchmark.

Stateless fuzzers like AFL are well-established and effective.

Research on stateful fuzzing faces unique challenges.

Abstract

Fuzzing has been proven extremely effective in finding vulnerabilities in software. When it comes to fuzz stateless systems, analysts have no doubts about the choice to make. In fact, among the plethora of stateless fuzzers devised in the last 20 years, AFL (with its descendants AFL++ and LibAFL) stood up for its effectiveness, speed and ability to find bugs. On the other hand, when dealing with stateful systems, it is not clear what is the best tool to use. In fact, the research community struggles to devise (and benchmark) effective and generic stateful fuzzers. In this short paper, we discuss the reasons that make stateful fuzzers difficult to devise and benchmark.