Validating Static Warnings via Testing Code Fragments

TL;DR

This paper introduces a method that automatically generates and tests code fragments from static warnings to validate their accuracy, improving bug detection in C programs.

Contribution

It presents a novel syntactic patching algorithm and an automated testing system that effectively validate static analysis warnings using fuzzers and symbolic execution.

Findings

Successfully built 68.5% of code fragments and generated 1003 test cases.

Identified 48 true positives and 27 false positives among static warnings.

Matched 4 CVEs and real-world bugs only triggered by our approach.

Abstract

Static analysis is an important approach for finding bugs and vulnerabilities in software. However, inspecting and confirming static warnings are challenging and time-consuming. In this paper, we present a novel solution that automatically generates test cases based on static warnings to validate true and false positives. We designed a syntactic patching algorithm that can generate syntactically valid, semantic preserving executable code fragments from static warnings. We developed a build and testing system to automatically test code fragments using fuzzers, KLEE and Valgrind. We evaluated our techniques using 12 real-world C projects and 1955 warnings from two commercial static analysis tools. We successfully built 68.5% code fragments and generated 1003 test cases. Through automatic testing, we identified 48 true positives and 27 false positives, and 205 likely false positives. We matched 4 CVE and real-world bugs using Helium, and they are only triggered by our tool but not other baseline tools. We found that testing code fragments is scalable and useful; it can trigger bugs that testing entire programs or testing procedures failed to trigger.