Using Cyber Terrain in Reinforcement Learning for Penetration Testing

TL;DR

This paper enhances reinforcement learning-based penetration testing by incorporating cyber terrain analysis into attack graphs, adding operational realism through concepts like obstacles and avenues of approach.

Contribution

It introduces methods to construct attack graphs using cyber terrain notions from IPB, improving the realism of RL agents in penetration testing scenarios.

Findings

Terrain analysis improves attack graph realism for RL

Firewalls modeled as obstacles influence reward and state dynamics

Enhanced attack graphs better reflect operational conditions

Abstract

Reinforcement learning (RL) has been applied to attack graphs for penetration testing, however, trained agents do not reflect reality because the attack graphs lack operational nuances typically captured within the intelligence preparation of the battlefield (IPB) that include notions of (cyber) terrain. In particular, current practice constructs attack graphs exclusively using the Common Vulnerability Scoring System (CVSS) and its components. We present methods for constructing attack graphs using notions from IPB on cyber terrain analysis of obstacles, avenues of approach, key terrain, observation and fields of fire, and cover and concealment. We demonstrate our methods on an example where firewalls are treated as obstacles and represented in (1) the reward space and (2) the state dynamics. We show that terrain analysis can be used to bring realism to attack graphs for RL.