Your Fix Is My Exploit: Enabling Comprehensive DL Library API Fuzzing with Large Language Models

TL;DR

This paper introduces DFUZZ, an LLM-driven fuzzing framework that significantly improves API coverage and bug detection in deep learning libraries like TensorFlow and PyTorch by leveraging LLM reasoning and test program synthesis.

Contribution

It presents a novel LLM-based approach for comprehensive API fuzzing in DL libraries, overcoming limitations of traditional fuzzers in complex, diverse API environments.

Findings

DFUZZ achieves higher API coverage than state-of-the-art fuzzers.

Discovered 37 bugs in TensorFlow and PyTorch, with 8 fixed.

Enhanced bug detection through LLM reasoning and test synthesis.

Abstract

Deep learning (DL) libraries, widely used in AI applications, often contain vulnerabilities like buffer overflows and use-after-free errors. Traditional fuzzing struggles with the complexity and API diversity of DL libraries such as TensorFlow and PyTorch, which feature over 1,000 APIs. Testing all these APIs is challenging due to complex inputs and varied usage patterns. While large language models (LLMs) show promise in code understanding and generation, existing LLM-based fuzzers lack deep knowledge of API edge cases and struggle with test input generation. To address this, we propose DFUZZ, an LLM-driven fuzzing approach for DL libraries. DFUZZ leverages two insights: (1) LLMs can reason about error-triggering edge cases from API code and apply this knowledge to untested APIs, and (2) LLMs can accurately synthesize test programs to automate API testing. By providing LLMs with a "white-box view" of APIs, DFUZZ enhances reasoning and generation for comprehensive fuzzing. Experimental results show that DFUZZ outperforms state-of-the-art fuzzers in API coverage for TensorFlow and PyTorch, uncovering 37 bugs, with 8 fixed and 19 under developer investigation.