Who Said CVE? How Vulnerability Identifiers Are Mentioned by Humans, Bots, and Agents in Pull Requests

TL;DR

This study investigates how vulnerability identifiers like CVE are used in GitHub pull requests by humans, bots, and agents, revealing distinct patterns and contexts of mention across different contributors.

Contribution

It provides the first comparative analysis of vulnerability ID mentions across human, bot, and agent contributions in pull requests, highlighting usage differences and contextual roles.

Findings

Bots account for 69.1% of mentions, mainly in automated updates.

Humans and agents mention identifiers less frequently but across more locations.

Different roles of identifiers in fixing, maintaining, and discussing vulnerabilities.

Abstract

Vulnerability identifiers such as CVE, CWE, and GHSA are standardised references to known software security issues, yet their use in practice is not well understood. This paper compares vulnerability ID use in GitHub pull requests authored by autonomous agents, bots, and human developers. Using the AIDev pop dataset and an augmented set of pull requests from the same repositories, we analyse who mentions vulnerability identifiers and where they appear. Bots account for around 69.1% of all mentions, usually adding few identifiers in pull request descriptions, while human and agent mentions are rarer but span more locations. Qualitative analysis shows that bots mainly reference identifiers in automated dependency updates and audits, whereas humans and agents use them to support fixes, maintenance, and discussion.