When Fuzzing Meets LLMs: Challenges and Opportunities

TL;DR

This paper explores the integration of Large Language Models into fuzzing, identifying key challenges, reviewing recent research, and proposing practical solutions validated through preliminary experiments on database management systems.

Contribution

It identifies five major challenges in LLM-assisted fuzzing, reviews current research, and offers actionable recommendations with preliminary validation.

Findings

Recommendations effectively address identified challenges

Preliminary evaluations show improved fuzzing performance

Challenges are widespread across recent studies

Abstract

Fuzzing, a widely-used technique for bug detection, has seen advancements through Large Language Models (LLMs). Despite their potential, LLMs face specific challenges in fuzzing. In this paper, we identified five major challenges of LLM-assisted fuzzing. To support our findings, we revisited the most recent papers from top-tier conferences, confirming that these challenges are widespread. As a remedy, we propose some actionable recommendations to help improve applying LLM in Fuzzing and conduct preliminary evaluations on DBMS fuzzing. The results demonstrate that our recommendations effectively address the identified challenges.