SAFuzz: Semantic-Guided Adaptive Fuzzing for LLM-Generated Code

TL;DR

SAFuzz is a novel adaptive fuzzing framework guided by large language models that efficiently detects algorithmic vulnerabilities in AI-generated code, improving precision and reducing testing time.

Contribution

It introduces a semantic-guided adaptive fuzzing approach integrating LLMs for behavioral diversification, resource allocation, and early stopping, enhancing vulnerability detection in AI-generated code.

Findings

Improves vulnerability discrimination precision from 77.9% to 85.7%.

Reduces testing time by 1.71x compared to SOTA GreenFuzz.

Combining with unit test methods increases bug detection recall to 79.5%.

Abstract

While AI-coding assistants accelerate software development, current testing frameworks struggle to keep pace with the resulting volume of AI-generated code. Traditional fuzzing techniques often allocate resources uniformly and lack semantic awareness of algorithmic vulnerability patterns, leading to inefficient resource usage and missed vulnerabilities. To address these limitations, we present a hybrid testing framework that leverages LLM-guided adaptive fuzzing to detect algorithmic vulnerabilities efficiently. Our system SAFuzz integrates prompt-based behavioral diversification, harness generation with problem-specific oracles, and an LLM-based predictor to enable adaptive resource allocation and dynamic early stopping. Evaluating SAFuzz on CSES algorithmic problems, we improve vulnerability discrimination precision from 77.9% to 85.7% and achieve a 1.71x reduction in time cost compared to SOTA GreenFuzz while maintaining comparable recall. We further observe that combining our approach with existing unit test generation methods yields complementary gains, increasing the bug detection recall from 67.3% to 79.5%.