Investigating Coverage Guided Fuzzing with Mutation Testing

TL;DR

This paper enhances coverage guided fuzzing by integrating mutation testing to prioritize bug detection over mere code coverage, leading to improved bug discovery in software testing.

Contribution

It introduces a novel approach that uses mutation scores as feedback to guide fuzzing towards bug detection rather than just increasing code coverage.

Findings

Improved bug detection compared to baseline Zest.

Enhanced code coverage in tested benchmarks.

Effective guidance towards interesting program states.

Abstract

Coverage guided fuzzing (CGF) is an effective testing technique which has detected hundreds of thousands of bugs from various software applications. It focuses on maximizing code coverage to reveal more bugs during fuzzing. However, a higher coverage does not necessarily imply a better fault detection capability. Triggering a bug involves not only exercising the specific program path but also reaching interesting program states in that path. In this paper, we use mutation testing to improve CGF in detecting bugs. We use mutation scores as feedback to guide fuzzing towards detecting bugs rather than just covering code. To evaluate our approach, we conduct a well-designed experiment on 5 benchmarks. We choose the state-of-the-art fuzzing technique Zest as baseline and construct two modified techniques on it using our approach. The experimental results show that our approach can improve CGF in both code coverage and bug detection.