Prompt Fuzzing for Fuzz Driver Generation

TL;DR

PromptFuzz is a novel coverage-guided prompt fuzzing approach that automatically generates effective fuzz drivers, significantly improving code coverage and bug detection in real-world libraries compared to existing tools.

Contribution

The paper introduces PromptFuzz, a new method combining prompt fuzzing with coverage guidance and novel techniques to generate high-quality fuzz drivers automatically.

Findings

PromptFuzz achieves 1.61-1.63x higher branch coverage than OSS-Fuzz and Hopper.

PromptFuzz detects 33 new bugs, with 30 confirmed by communities.

The approach outperforms state-of-the-art fuzz driver generation tools.

Abstract

Crafting high-quality fuzz drivers not only is time-consuming but also requires a deep understanding of the library. However, the state-of-the-art automatic fuzz driver generation techniques fall short of expectations. While fuzz drivers derived from consumer code can reach deep states, they have limited coverage. Conversely, interpretative fuzzing can explore most API calls but requires numerous attempts within a large search space. We propose PromptFuzz, a coverage-guided fuzzer for prompt fuzzing that iteratively generates fuzz drivers to explore undiscovered library code. To explore API usage in fuzz drivers during prompt fuzzing, we propose several key techniques: instructive program generation, erroneous program validation, coverage-guided prompt mutation, and constrained fuzzer scheduling. We implemented PromptFuzz and evaluated it on 14 real-world libraries. Compared with OSS-Fuzz and Hopper (the state-of-the-art fuzz driver generation tool), fuzz drivers generated by PromptFuzz achieved 1.61 and 1.63 times higher branch coverage than those by OSS-Fuzz and Hopper, respectively. Moreover, the fuzz drivers generated by PromptFuzz detected 33 genuine, new bugs out of a total of 49 crashes, out of which 30 bugs have been confirmed by their respective communities.