VFDelta: A Framework for Detecting Silent Vulnerability Fixes by Enhancing Code Change Learning

TL;DR

VFDelta is a novel framework that effectively detects silent vulnerability fixes in open source software by learning nuanced code change representations through joint training of embedding and classification models, significantly outperforming existing methods.

Contribution

The paper introduces VFDelta, a new framework that captures fine-grain code changes using independent embeddings and joint training, improving vulnerability fix detection accuracy.

Findings

VFDelta achieves up to 0.33 F1 score, outperforming state-of-the-art methods.

It improves CostEffort@5 by 7.1%, demonstrating better early detection.

Expanded dataset and temporal split show VFDelta's robustness in real-world scenarios.

Abstract

Vulnerability fixes in open source software (OSS) usually follow the coordinated vulnerability disclosure model and are silently fixed. This delay can expose OSS users to risks as malicious parties might exploit the software before fixes are publicly known. Therefore, it is important to identify vulnerability fixes early and automatically. Existing methods classify vulnerability fixes by learning code change representations from commits, typically by concatenating code changes, which does not effectively highlight nuanced differences. Additionally, previous approaches fine-tune code embedding models and classification models separately, which limits overall effectiveness. We propose VFDelta, a lightweight yet effective framework that embeds code before and after changes using independent models with surrounding code as context. By performing element-wise subtraction on these embeddings, we capture fine-grain changes. Our architecture allows joint training of embedding and classification models, optimizing overall performance. Experiments demonstrate that VFDelta achieves up to 0.33 F1 score and 0.63 CostEffort@5, improving over state-of-the-art methods by 77.4% and 7.1%, respectively. Ablation analysis confirms the importance of our code change representation in capturing small changes. We also expanded the dataset and introduced a temporal split to simulate real-world scenarios; VFDelta significantly outperforms baselines VulFixMiner and MiDas across all metrics in this setting.