Combining BMC and Fuzzing Techniques for Finding Software Vulnerabilities in Concurrent Programs

TL;DR

This paper introduces EBF, a novel ensemble technique combining Bounded Model Checking and Gray-Box Fuzzing, including a new concurrency-aware fuzzer, to improve vulnerability detection in concurrent programs.

Contribution

The paper presents EBF, integrating BMC and a new open-source concurrency-aware GBF, enhancing vulnerability detection in concurrent software beyond existing methods.

Findings

EBF finds 14.9% more verification witnesses than pure BMC.

OpenGBF detects 24.2% of vulnerabilities, outperforming non-concurrency-aware fuzzers.

EBF successfully detects real-world vulnerabilities, including data races in open-source software.

Abstract

Finding software vulnerabilities in concurrent programs is a challenging task due to the size of the state-space exploration, as the number of interleavings grows exponentially with the number of program threads and statements. We propose and evaluate EBF (Ensembles of Bounded Model Checking with Fuzzing) -- a technique that combines Bounded Model Checking (BMC) and Gray-Box Fuzzing (GBF) to find software vulnerabilities in concurrent programs. Since there are no publicly-available GBF tools for concurrent code, we first propose OpenGBF -- a new open-source concurrency-aware gray-box fuzzer that explores different thread schedules by instrumenting the code under test with random delays. Then, we build an ensemble of a BMC tool and OpenGBF in the following way. On the one hand, when the BMC tool in the ensemble returns a counterexample, we use it as a seed for OpenGBF, thus increasing the likelihood of executing paths guarded by complex mathematical expressions. On the other hand, we aggregate the outcomes of the BMC and GBF tools in the ensemble using a decision matrix, thus improving the accuracy of EBF. We evaluate EBF against state-of-the-art pure BMC tools and show that it can generate up to 14.9% more correct verification witnesses than the corresponding BMC tools alone. Furthermore, we demonstrate the efficacy of OpenGBF, by showing that it can find 24.2% of the vulnerabilities in our evaluation suite, while non-concurrency-aware GBF tools can only find 0.55%. Finally, thanks to our concurrency-aware OpenGBF, EBF detects a data race in the open-source wolfMqtt library and reproduces known bugs in several other real-world programs, which demonstrates its effectiveness in finding vulnerabilities in real-world software.