MoCo: Fuzzing Deep Learning Libraries via Assembling Code

TL;DR

MoCo introduces a novel code assembly-based fuzzing approach for DL libraries, improving bug detection accuracy and efficiency, and successfully discovering numerous new bugs in popular DL frameworks.

Contribution

MoCo's innovative code assembly and mutation strategy enhances fuzzing diversity and precision, addressing limitations of prior DL library testing methods.

Findings

Detected 64 new bugs across three DL libraries

Confirmed 51 bugs and 13 fixed by developers

Demonstrated high efficiency and effectiveness of MoCo

Abstract

The rapidly developing deep learning (DL) techniques have been applied in software systems with various application scenarios. However, they could also pose new safety threats with potentially serious consequences, especially in safety-critical domains. DL libraries serve as the underlying foundation for DL systems, and bugs in them can have unpredictable impacts that directly affect the behaviors of DL systems. Previous research on fuzzing DL libraries still has limitations in the diversity of test inputs, the construction of test oracles, and the precision of detection. In this paper, we propose MoCo, a novel fuzzing testing method for DL libraries via assembling code. MoCo first disassembles the seed code file to obtain the template and code blocks, and then employs code block mutation operators (e.g., API replacement, random generation, and boundary checking) to generate more new code blocks adapted to the template. By inserting context-appropriate code blocks into the template step by step, MoCo can generate a tree of code files with intergenerational relations. According to the derivation relations in this tree and the applied mutation operators, we construct the test oracle based on the execution state consistency. Since the granularity of code assembly and mutation is controlled rather than randomly divergent, we can quickly pinpoint the lines of code where the bugs are located and the corresponding triggering conditions. We conduct a comprehensive experiment to evaluate the efficiency and effectiveness of MoCo using three widely-used DL libraries (i.e., TensorFlow, PyTorch, and Jittor). During the experiment, MoCo detects 64 new bugs of four types in three DL libraries, where 51 bugs have been confirmed, and 13 bugs have been fixed by developers.