SoK: Automated Vulnerability Repair: Methods, Tools, and Assessments

TL;DR

This paper surveys automated vulnerability repair methods, introduces a new benchmark dataset for C/C++ vulnerabilities, and evaluates existing tools, highlighting current capabilities and future research directions.

Contribution

It systematizes AVR methods, creates the first comprehensive C/C++ vulnerability repair benchmark, and provides an extensive evaluation of existing AVR tools for C/C++ and Java.

Findings

Seven C/C++ AVR tools evaluated on Vul4C dataset.

Two Java AVR tools evaluated on Vul4J dataset.

Discussion of future research directions.

Abstract

The increasing complexity of software has led to the steady growth of vulnerabilities. Vulnerability repair investigates how to fix software vulnerabilities. Manual vulnerability repair is labor-intensive and time-consuming because it relies on human experts, highlighting the importance of Automated Vulnerability Repair (AVR). In this SoK, we present the systematization of AVR methods through the three steps of AVR workflow: vulnerability analysis, patch generation, and patch validation. We assess AVR tools for C/C++ and Java programs as they have been widely studied by the community. Since existing AVR tools for C/C++ programs are evaluated with different datasets, which often consist of a few vulnerabilities, we construct the first C/C++ vulnerability repair benchmark dataset, dubbed Vul4C, which contains 144 vulnerabilities as well as their exploits and patches. We use Vul4C to evaluate seven AVR tools for C/C++ programs and use the third-party Vul4J dataset to evaluate two AVR tools for Java programs. We also discuss future research directions.