Learning Intrusion Prevention Policies through Optimal Stopping

TL;DR

This paper introduces a reinforcement learning approach to automated intrusion prevention by modeling it as an optimal stopping problem, resulting in threshold-based policies that are near-optimal in simulated environments.

Contribution

It formulates intrusion prevention as an optimal stopping problem and demonstrates how to learn effective threshold-based policies via reinforcement learning.

Findings

Learned policies are close to optimal

Policies can be expressed as thresholds

Reinforcement learning effectively approximates optimal policies

Abstract

We study automated intrusion prevention using reinforcement learning. In a novel approach, we formulate the problem of intrusion prevention as an optimal stopping problem. This formulation allows us insight into the structure of the optimal policies, which turn out to be threshold based. Since the computation of the optimal defender policy using dynamic programming is not feasible for practical cases, we approximate the optimal policy through reinforcement learning in a simulation environment. To define the dynamics of the simulation, we emulate the target infrastructure and collect measurements. Our evaluations show that the learned policies are close to optimal and that they indeed can be expressed using thresholds.