On the Challenges of Fuzzing Techniques via Large Language Models

TL;DR

This paper reviews the emerging use of large language models to automate fuzzing tests, highlighting their potential to improve software security testing despite current challenges.

Contribution

It is the first comprehensive overview of how LLMs are applied to fuzzing, analyzing recent developments and future prospects in this intersection.

Findings

LLMs show promise in automating fuzzing test generation.

Current techniques face challenges in automation and reliability.

Potential for widespread adoption in automated software testing.

Abstract

In the modern era where software plays a pivotal role, software security and vulnerability analysis are essential for secure software development. Fuzzing test, as an efficient and traditional software testing method, has been widely adopted across various domains. Meanwhile, the rapid development in Large Language Models (LLMs) has facilitated their application in the field of software testing, demonstrating remarkable performance. As existing fuzzing test techniques are not fully automated and software vulnerabilities continue to evolve, there is a growing interest in leveraging large language models to generate fuzzing test. In this paper, we present a systematic overview of the developments that utilize large language models for the fuzzing test. To our best knowledge, this is the first work that covers the intersection of three areas, including LLMs, fuzzing test, and fuzzing test generated based on LLMs. A statistical analysis and discussion of the literature are conducted by summarizing the state-of-the-art methods up to date of the submission. Our work also investigates the potential for widespread deployment and application of fuzzing test techniques generated by LLMs in the future, highlighting their promise for advancing automated software testing practices.