S$^2$F: Principled Hybrid Testing With Fuzzing, Symbolic Execution, and Sampling

TL;DR

S$^2$F is a hybrid testing tool that effectively combines fuzzing, symbolic execution, and sampling to improve code coverage and crash discovery, outperforming existing tools on real-world programs.

Contribution

The paper introduces a novel hybrid testing architecture and principles that better integrate symbolic execution and sampling, leading to improved testing efficiency.

Findings

Achieves 6.14% higher edge coverage on average.

Discovers 32.6% more crashes than state-of-the-art tools.

Uncovers three previously unknown crashes in real-world programs.

Abstract

Hybrid testing that integrates fuzzing, symbolic execution, and sampling has demonstrated superior testing efficiency compared to individual techniques. However, the state-of-the-art (SOTA) hybrid testing tools do not fully exploit the capabilities of symbolic execution and sampling in two key aspects. First, the SOTA hybrid testing tools employ tailored symbolic execution engines that tend to over-prune branches, leading to considerable time wasted waiting for seeds from the fuzzer and missing opportunities to discover crashes. Second, existing methods do not apply sampling to the appropriate branches and therefore cannot utilize the full capability of sampling. To address these two limitations, we propose a novel hybrid testing architecture that combines the precision of conventional symbolic execution with the scalability of tailored symbolic execution engines. Based on this architecture, we propose several principles for combining fuzzing, symbolic execution, and sampling. We implement our method in a hybrid testing tool S$^2$F. To evaluate its effectiveness, we conduct extensive experiments on 15 real-world programs. Experimental results demonstrate that S$^2$F outperforms the SOTA tool, achieving an average improvement of 6.14% in edge coverage and 32.6% in discovered crashes. Notably, our tool uncovers three previously unknown crashes in real-world programs.