Automated software vulnerability detection with machine learning

TL;DR

This paper introduces a machine learning approach for detecting security vulnerabilities in C and C++ code, leveraging large datasets and combining deep learning with traditional models to improve detection accuracy.

Contribution

It presents a data-driven vulnerability detection method that combines deep neural networks with tree-based models, outperforming traditional approaches.

Findings

Deep models combined with random forests yield best performance.

Source-based models outperform build artifact-based models.

Highest model achieves AUPRC of 0.49 and AUROC of 0.87.

Abstract

Thousands of security vulnerabilities are discovered in production software each year, either reported publicly to the Common Vulnerabilities and Exposures database or discovered internally in proprietary code. Vulnerabilities often manifest themselves in subtle ways that are not obvious to code reviewers or the developers themselves. With the wealth of open source code available for analysis, there is an opportunity to learn the patterns of bugs that can lead to security vulnerabilities directly from data. In this paper, we present a data-driven approach to vulnerability detection using machine learning, specifically applied to C and C++ programs. We first compile a large dataset of hundreds of thousands of open-source functions labeled with the outputs of a static analyzer. We then compare methods applied directly to source code with methods applied to artifacts extracted from the build process, finding that source-based models perform better. We also compare the application of deep neural network models with more traditional models such as random forests and find the best performance comes from combining features learned by deep models with tree-based models. Ultimately, our highest performing model achieves an area under the precision-recall curve of 0.49 and an area under the ROC curve of 0.87.