PoCGen: Generating Proof-of-Concept Exploits for Vulnerabilities in Npm Packages

TL;DR

PoCGen is an innovative system that automatically generates and validates proof-of-concept exploits for npm package vulnerabilities by combining large language models with static and dynamic analysis, significantly aiding security patching efforts.

Contribution

PoCGen introduces a novel hybrid approach that integrates LLMs, static analysis, and dynamic validation to automate PoC exploit generation for npm vulnerabilities.

Findings

Successfully generates exploits for 77% of vulnerabilities in SecBench.js

Outperforms recent baseline by 45 percentage points

Generates exploits at an average cost of $0.02 each

Abstract

Security vulnerabilities in software packages are a significant concern for developers and users alike. Patching these vulnerabilities in a timely manner is crucial to restoring the integrity and security of software systems. However, previous work has shown that vulnerability reports often lack proof-of-concept (PoC) exploits, which are essential for fixing the vulnerability, testing patches, and avoiding regressions. Creating a PoC exploit is challenging because vulnerability reports are informal and often incomplete, and because it requires a detailed understanding of how inputs passed to potentially vulnerable APIs may reach security-relevant sinks. In this paper, we present PoCGen, a novel approach to autonomously generate and validate PoC exploits for vulnerabilities in npm packages. The approach is the first to address this task by combining the complementary strengths of large language models (LLMs), e.g., to understand informal vulnerability reports, with static analysis, e.g., to identify taint paths, and dynamic analysis, e.g., to validate generated exploits. PoCGen successfully generates exploits for 77% of the vulnerabilities in the SecBench$.$js dataset. This success rate significantly outperforms a recent baseline (by 45 absolute percentage points), while imposing an average cost of only $0.02 per generated exploit. Moreover, PoCGen generates six successful exploits for recent real-world vulnerabilities, five of which are now included in their respective vulnerability reports.