Deep VULMAN: A Deep Reinforcement Learning-Enabled Cyber Vulnerability Management Framework

TL;DR

Deep VULMAN introduces a deep reinforcement learning framework combined with integer programming to optimize cyber vulnerability mitigation, outperforming existing methods by effectively prioritizing vulnerabilities under resource constraints and uncertainty.

Contribution

It presents a novel sequential decision-making framework integrating deep reinforcement learning and integer programming for cyber vulnerability management, addressing uncertainty and resource allocation.

Findings

Outperforms current methods in vulnerability prioritization.

Effective in resource allocation under uncertainty.

Validated on simulated and real-world data over one year.

Abstract

Cyber vulnerability management is a critical function of a cybersecurity operations center (CSOC) that helps protect organizations against cyber-attacks on their computer and network systems. Adversaries hold an asymmetric advantage over the CSOC, as the number of deficiencies in these systems is increasing at a significantly higher rate compared to the expansion rate of the security teams to mitigate them in a resource-constrained environment. The current approaches are deterministic and one-time decision-making methods, which do not consider future uncertainties when prioritizing and selecting vulnerabilities for mitigation. These approaches are also constrained by the sub-optimal distribution of resources, providing no flexibility to adjust their response to fluctuations in vulnerability arrivals. We propose a novel framework, Deep VULMAN, consisting of a deep reinforcement learning agent and an integer programming method to fill this gap in the cyber vulnerability management process. Our sequential decision-making framework, first, determines the near-optimal amount of resources to be allocated for mitigation under uncertainty for a given system state and then determines the optimal set of prioritized vulnerability instances for mitigation. Our proposed framework outperforms the current methods in prioritizing the selection of important organization-specific vulnerabilities, on both simulated and real-world vulnerability data, observed over a one-year period.