Noisy Label Learning for Security Defects

TL;DR

This paper introduces robust noisy label learning methods for security defect prediction, addressing label noise issues in vulnerability datasets to improve predictive performance.

Contribution

It proposes a novel two-stage noise cleaning approach for vulnerability prediction, enhancing model accuracy despite noisy labels.

Findings

Improved AUC and recall by up to 8.9% and 23.4% with the proposed method.

Demonstrated effectiveness of noisy label learning in security analytics.

Discussed challenges in achieving performance upper bounds with label noise.

Abstract

Data-driven software engineering processes, such as vulnerability prediction heavily rely on the quality of the data used. In this paper, we observe that it is infeasible to obtain a noise-free security defect dataset in practice. Despite the vulnerable class, the non-vulnerable modules are difficult to be verified and determined as truly exploit free given the limited manual efforts available. It results in uncertainty, introduces labeling noise in the datasets and affects conclusion validity. To address this issue, we propose novel learning methods that are robust to label impurities and can leverage the most from limited label data; noisy label learning. We investigate various noisy label learning methods applied to software vulnerability prediction. Specifically, we propose a two-stage learning method based on noise cleaning to identify and remediate the noisy samples, which improves AUC and recall of baselines by up to 8.9% and 23.4%, respectively. Moreover, we discuss several hurdles in terms of achieving a performance upper bound with semi-omniscient knowledge of the label noise. Overall, the experimental results show that learning from noisy labels can be effective for data-driven software and security analytics.