The Benefits of Vulnerability Discovery and Bug Bounty Programs: Case Studies of Chromium and Firefox

TL;DR

This study analyzes bug-bounty programs for Chromium and Firefox, showing they improve security by making vulnerabilities harder to discover and highlighting differences in vulnerabilities found internally, externally, and exploited by attackers.

Contribution

It introduces a novel metric for vulnerability discovery difficulty and compares vulnerability types across internal, external, and threat actor findings.

Findings

Vulnerability discovery becomes harder due to bug-bounty programs.

External bug hunters find different vulnerabilities than internal teams.

Bug-bounty programs help prevent vulnerabilities from being exploited by attackers.

Abstract

Recently, bug-bounty programs have gained popularity and become a significant part of the security culture of many organizations. Bug-bounty programs enable organizations to enhance their security posture by harnessing the diverse expertise of crowds of external security experts (i.e., bug hunters). Nonetheless, quantifying the benefits of bug-bounty programs remains elusive, which presents a significant challenge for managing them. Previous studies focused on measuring their benefits in terms of the number of vulnerabilities reported or based on the properties of the reported vulnerabilities, such as severity or exploitability. However, beyond these inherent properties, the value of a report also depends on the probability that the vulnerability would be discovered by a threat actor before an internal expert could discover and patch it. In this paper, we present a data-driven study of the Chromium and Firefox vulnerability-reward programs. First, we estimate the difficulty of discovering a vulnerability using the probability of rediscovery as a novel metric. Our findings show that vulnerability discovery and patching provide clear benefits by making it difficult for threat actors to find vulnerabilities; however, we also identify opportunities for improvement, such as incentivizing bug hunters to focus more on development releases. Second, we compare the types of vulnerabilities that are discovered internally vs. externally and those that are exploited by threat actors. We observe significant differences between vulnerabilities found by external bug hunters, internal security teams, and external threat actors, which indicates that bug-bounty programs provide an important benefit by complementing the expertise of internal teams, but also that external hunters should be incentivized more to focus on the types of vulnerabilities that are likely to be exploited by threat actors.