Optimizing Cyber Defense in Dynamic Active Directories through Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning framework for dynamic Active Directory cybersecurity, employing a Stackelberg game model to develop scalable, adaptive attack and defense strategies that improve robustness against cyber threats.

Contribution

It presents a novel RL-based attack and defense approach tailored for dynamic AD systems, incorporating an RL Training Facilitator for scalable training on large graphs.

Findings

Enhanced defense strategies against dynamic AD attacks

Scalable RL training method for large-scale graphs

Improved robustness of AD systems in simulations

Abstract

This paper addresses a significant gap in Autonomous Cyber Operations (ACO) literature: the absence of effective edge-blocking ACO strategies in dynamic, real-world networks. It specifically targets the cybersecurity vulnerabilities of organizational Active Directory (AD) systems. Unlike the existing literature on edge-blocking defenses which considers AD systems as static entities, our study counters this by recognizing their dynamic nature and developing advanced edge-blocking defenses through a Stackelberg game model between attacker and defender. We devise a Reinforcement Learning (RL)-based attack strategy and an RL-assisted Evolutionary Diversity Optimization-based defense strategy, where the attacker and defender improve each other strategy via parallel gameplay. To address the computational challenges of training attacker-defender strategies on numerous dynamic AD graphs, we propose an RL Training Facilitator that prunes environments and neural networks to eliminate irrelevant elements, enabling efficient and scalable training for large graphs. We extensively train the attacker strategy, as a sophisticated attacker model is essential for a robust defense. Our empirical results successfully demonstrate that our proposed approach enhances defender's proficiency in hardening dynamic AD graphs while ensuring scalability for large-scale AD.