GPU-Fuzz: Finding Memory Errors in Deep Learning Frameworks

TL;DR

GPU-Fuzz is a novel testing tool that systematically finds memory errors in deep learning frameworks by modeling operator parameters as constraints and using a solver to generate targeted test cases.

Contribution

GPU-Fuzz introduces a formal constraint-based fuzzing approach to efficiently detect memory errors in GPU-accelerated deep learning frameworks.

Findings

Uncovered 13 previously unknown bugs in popular DL frameworks

Demonstrated effectiveness of constraint-based fuzzing for GPU kernel testing

Applicable to multiple frameworks like PyTorch, TensorFlow, PaddlePaddle

Abstract

GPU memory errors are a critical threat to deep learning (DL) frameworks, leading to crashes or even security issues. We introduce GPU-Fuzz, a fuzzer locating these issues efficiently by modeling operator parameters as formal constraints. GPU-Fuzz utilizes a constraint solver to generate test cases that systematically probe error-prone boundary conditions in GPU kernels. Applied to PyTorch, TensorFlow, and PaddlePaddle, we uncovered 13 unknown bugs, demonstrating the effectiveness of GPU-Fuzz in finding memory errors.