Toward Scalable Automated Repository-Level Datasets for Software Vulnerability Detection

TL;DR

This paper introduces an automated method to generate large-scale, realistic, repository-level datasets for software vulnerability detection by injecting vulnerabilities into real repositories and creating reproducible exploits, enhancing training and evaluation.

Contribution

It presents a novel automated benchmark generator that scales vulnerability dataset creation and explores adversarial co-evolution to improve detection robustness.

Findings

Automated injection of vulnerabilities into real repositories.

Generation of reproducible proof-of-vulnerability exploits.

Enhanced dataset scale and realism for vulnerability detection.

Abstract

Software vulnerabilities continue to grow in volume and remain difficult to detect in practice. Although learning-based vulnerability detection has progressed, existing benchmarks are largely function-centric and fail to capture realistic, executable, interprocedural settings. Recent repo-level security benchmarks demonstrate the importance of realistic environments, but their manual curation limits scale. This doctoral research proposes an automated benchmark generator that injects realistic vulnerabilities into real-world repositories and synthesizes reproducible proof-of-vulnerability (PoV) exploits, enabling precisely labeled datasets for training and evaluating repo-level vulnerability detection agents. We further investigate an adversarial co-evolution loop between injection and detection agents to improve robustness under realistic constraints.