Boosting Vulnerability Detection of LLMs via Curriculum Preference Optimization with Synthetic Reasoning Data

TL;DR

This paper introduces ReVD, a novel framework that enhances vulnerability detection in large language models by synthesizing reasoning data and optimizing vulnerability-specific preferences, leading to state-of-the-art results.

Contribution

ReVD is the first framework to synthesize high-quality reasoning data and apply curriculum preference optimization for improved vulnerability detection in LLMs.

Findings

ReVD achieves 12.24%-22.77% accuracy improvement on PrimeVul and SVEN datasets.

ReVD outperforms existing methods in vulnerability detection tasks.

Synthetic reasoning data enhances the model's ability to recognize vulnerability patterns.

Abstract

Large language models (LLMs) demonstrate considerable proficiency in numerous coding-related tasks; however, their capabilities in detecting software vulnerabilities remain limited. This limitation primarily stems from two factors: (1) the absence of reasoning data related to vulnerabilities, which hinders the models' ability to capture underlying vulnerability patterns; and (2) their focus on learning semantic representations rather than the reason behind them, thus failing to recognize semantically similar vulnerability samples. Furthermore, the development of LLMs specialized in vulnerability detection is challenging, particularly in environments characterized by the scarcity of high-quality datasets. In this paper, we propose a novel framework ReVD that excels at mining vulnerability patterns through reasoning data synthesizing and vulnerability-specific preference optimization. Specifically, we construct forward and backward reasoning processes for vulnerability and corresponding fixed code, ensuring the synthesis of high-quality reasoning data. Moreover, we design the triplet supervised fine-tuning followed by curriculum online preference optimization for enabling ReVD to better understand vulnerability patterns. The extensive experiments conducted on PrimeVul and SVEN datasets demonstrate that ReVD sets new state-of-the-art for LLM-based software vulnerability detection, e.g., 12.24\%-22.77\% improvement in the accuracy. The source code and data are available at https://github.com/Xin-Cheng-Wen/PO4Vul.