Adversarial Deep Reinforcement Learning based Adaptive Moving Target Defense

TL;DR

This paper introduces an adaptive moving target defense strategy using multi-agent reinforcement learning to optimize system security against strategic adversaries.

Contribution

It formulates MTD as a multi-agent game and applies a double oracle RL algorithm to derive optimal defense policies.

Findings

The framework effectively finds optimal MTD policies.

The approach increases attacker uncertainty and attack cost.

Experimental results demonstrate the method's effectiveness.

Abstract

Moving target defense (MTD) is a proactive defense approach that aims to thwart attacks by continuously changing the attack surface of a system (e.g., changing host or network configurations), thereby increasing the adversary's uncertainty and attack cost. To maximize the impact of MTD, a defender must strategically choose when and what changes to make, taking into account both the characteristics of its system as well as the adversary's observed activities. Finding an optimal strategy for MTD presents a significant challenge, especially when facing a resourceful and determined adversary who may respond to the defender's actions. In this paper, we propose a multi-agent partially-observable Markov Decision Process model of MTD and formulate a two-player general-sum game between the adversary and the defender. Based on an established model of adaptive MTD, we propose a multi-agent reinforcement learning framework based on the double oracle algorithm to solve the game. In the experiments, we show the effectiveness of our framework in finding optimal policies.