An Exploratory Study on Regression Vulnerabilities

TL;DR

This study investigates how security regressions occur in software development, revealing that developers often overlook security during bug fixes and rely on tools that detect only about 30% of vulnerabilities, highlighting the need for improved security practices.

Contribution

It provides an exploratory analysis of regression vulnerabilities in Mozilla, emphasizing the gaps in developer awareness and tool effectiveness during bug fixes.

Findings

Security is not discussed during bug fixes.

Developers rely on tools that detect ~30% of regressions.

Security regressions are often introduced due to complexity and pressure.

Abstract

Background: Security regressions are vulnerabilities introduced in a previously unaffected software system. They often happen as a result of source code changes (e.g., a bug fix) and can have severe effects. Aims: To increase the understanding of security regressions. This is an important step in developing secure software engineering. Method: We perform an exploratory, mixed-method case study of Mozilla. First, we analyze 78 regression vulnerabilities and 72 bug reports where a bug fix introduced a regression vulnerability at Mozilla. We investigate how developers interact in these bug reports, how they perform the changes, and under what conditions they introduce regression vulnerabilities. Second, we conduct five semi-structured interviews with as many Mozilla developers involved in the vulnerability-inducing bug fixes. Results: Software security is not discussed during bug fixes. Developers' main concerns are the complexity of the bug at hand and the community pressure to fix it. Moreover, developers do not to worry about regression vulnerabilities and assume tools will detect them. Indeed, dynamic analysis tools helped finding around 30% of regression vulnerabilities at Mozilla. Conclusions: These results provide evidence that, although tool support helps identify regression vulnerabilities, it may not be enough to ensure security during bug fixes. Furthermore, our results call for further work on the security tooling support and how to integrate them during bug fixes. Data and materials: https://doi.org/10.5281/zenodo.6792317