Markov Decision Process For Automatic Cyber Defense

TL;DR

This paper introduces an automated cyber defense framework using Markov Decision Processes and Q-learning to optimize defense strategies against cyber-attacks, reducing risk and increasing adaptability.

Contribution

It presents a novel MDP-based framework that automates defense deployment in networked systems, incorporating real network data and learning optimal actions through Q-learning.

Findings

The model effectively reduces cyber attack risks in simulations.

It demonstrates flexibility with different Q-learning parameters.

The framework automates defense decisions based on real network data.

Abstract

It is challenging for a security analyst to detect or defend against cyber-attacks. Moreover, traditional defense deployment methods require the security analyst to manually enforce the defenses in the presence of uncertainties about the defense to deploy. As a result, it is essential to develop an automated and resilient defense deployment mechanism to thwart the new generation of attacks. In this paper, we propose a framework based on Markov Decision Process (MDP) and Q-learning to automatically generate optimal defense solutions for networked system states. The framework consists of four phases namely; the model initialization phase, model generation phase, Q-learning phase, and the conclusion phase. The proposed model collects real network information as inputs and then builds them into structural data. We implement a Q-learning process in the model to learn the quality of a defense action in a particular state. To investigate the feasibility of the proposed model, we perform simulation experiments and the result reveals that the model can reduce the risk of network systems from cyber attacks. Furthermore, the experiment shows that the model has shown a certain level of flexibility when different parameters are used for Q-learning.