VULPO: Context-Aware Vulnerability Detection via On-Policy LLM Optimization

TL;DR

VULPO introduces an on-policy reinforcement learning framework for vulnerability detection that effectively incorporates contextual repository information, significantly outperforming existing methods and enabling more accurate, comprehensive vulnerability analysis.

Contribution

The paper presents VULPO, a novel on-policy LLM reinforcement learning approach with a new dataset and multi-dimensional reward structure for context-aware vulnerability detection.

Findings

VULPO-4B outperforms prompt engineering and off-policy baselines.

Achieves 85% F1 improvement over Qwen3-4B.

Comparable to a 150x larger model, DeepSeek-R1-0528.

Abstract

The widespread reliance on open-source software dramatically increases the risk of vulnerability exploitation, underscoring the need for effective and scalable vulnerability detection (VD). Existing VD techniques, whether traditional machine learning-based or LLM-based approaches like prompt engineering, supervised fine-tuning, or off-policy preference optimization, remain fundamentally limited in their ability to perform context-aware analysis: They depend on fixed inputs or static preference datasets, cannot adaptively explore repository-level dependencies, and are constrained by function-level benchmarks that overlook critical vulnerability context. This paper introduces Vulnerability-Adaptive Policy Optimization (VULPO), an on-policy LLM reinforcement learning framework for context-aware VD. To support training and evaluation, we first construct ContextVul, a new dataset that augments high-quality function-level samples with lightweight method to extract repository-level context information. We then design multi-dimensional reward structuring that jointly captures prediction correctness, vulnerability localization accuracy, and the semantic relevance of vulnerability analysis, thereby guiding the model toward comprehensive contextual reasoning. To address the asymmetric difficulty of different vulnerability cases and mitigate reward hacking, VULPO incorporates label-level and sample-level difficulty-adaptive reward scaling, encouraging the model to explore challenging cases while maintaining balanced reward distribution. Extensive experiments demonstrate the superiority of our VULPO framework in context-aware VD: Our VULPO-4B substantially outperforms existing VD baselines based on prompt engineering and off-policy optimization, improving F1 by 85% over Qwen3-4B and achieving performance comparable to a 150x larger-scale model, DeepSeek-R1-0528.