MAS-SZZ: Multi-Agentic SZZ Algorithm for Vulnerability-Inducing Commit Identification

TL;DR

MAS-SZZ introduces a multi-agent collaboration approach to improve vulnerability-inducing commit identification, significantly outperforming existing algorithms in accuracy across multiple datasets.

Contribution

The paper presents MAS-SZZ, a novel multi-agentic SZZ algorithm that enhances vulnerability commit tracing through structured prompts and agent collaboration.

Findings

MAS-SZZ achieves up to 65.22% higher F1-score than state-of-the-art methods.

MAS-SZZ effectively localizes vulnerability-related statements using structured step-forward prompting.

Extensive experiments validate MAS-SZZ's superior performance across datasets and programming languages.

Abstract

Accurate vulnerability-inducing commit identification serves as a foundation for a series of software security tasks, such as vulnerability detection and affected version analysis. A straightforward solution is the SZZ algorithm, which traces back through the code history to identify the earliest commit that modify the vulnerable code. Unfortunately, neither the customized V-SZZ nor state-of-the-art LLM4SZZ perform satisfactorily due to the incorrect anchor selection and inadequate backtracking capability, making them far beyond a reliable usage in practice. To overcome these challenges, we propose a multi-agentic SZZ algorithm, named MAS-SZZ, that facilitates the identification of vulnerability-inducing commits through collaboration among agents. Specifically, given a CVE description and its corresponding fixing commit, MAS-SZZ summarizes the root cause of the vulnerability and employs a structured step-forward prompting strategy to localize vulnerability-related statements based on the change intent of each patch hunk. These vulnerable statements serve as anchors from which MAS-SZZ autonomously traces backward through the repository's history to find the commit that first introduced the vulnerability. Extensive experiments show that MAS-SZZ outperforms the state-of-the-art baselines across datasets and programming languages, achieving F1-score gains of up to 65.22% over the best-performing SZZ algorithm.