Memory Error Detection in Security Testing

TL;DR

This study analyzes memory error detection in security testing across multiple C/C++ projects, revealing detection frequency, developer response times, and the prevalence of memory-related CVEs, highlighting gaps in static analysis effectiveness.

Contribution

It provides an empirical analysis of memory error detection and developer response in security testing, and evaluates the detection capability of static analysis tools for memory-related vulnerabilities.

Findings

Memory alerts are frequently detected during static analysis.

Median fix time for memory alerts is 77 days.

40% of CVEs are memory-related, with higher severity.

Abstract

We study 10 C/C++ projects that have been using a static analysis security testing tool. We analyze the historical scan reports generated by the tool and study how frequently memory-related alerts appeared. We also studied the subsequent developer action on those alerts. We also look at the CVEs published for these projects within the study timeline and investigate how many of them are memory related. Moreover, for one of this project, Linux, we investigate if the involved flaws in the CVE were identified by the studied security tool when they were first introduced in the code. We found memory related alerts to be frequently detected during static analysis security testing. However, based on how actively the project developers are monitoring the tool alerts, these errors can take years to get fixed. For the ten studied projects, we found a median lifespan of 77 days before memory alerts get fixed. We also find that around 40% of the published CVEs for the studied C/C++ projects are related to memory. These memory CVEs have higher CVSS severity ratings and likelihood of having an exploit script public than non-memory CVEs. We also found only 2.5% Linux CVEs were possibly detected during static analysis security testing.