Network Environment Design for Autonomous Cyberdefense

TL;DR

This paper presents a novel framework for designing network environments to develop and evaluate reinforcement learning agents for autonomous cybersecurity defense, emphasizing robustness against sophisticated adversarial attacks.

Contribution

It introduces a new approach and software framework for creating realistic, adaptable network simulation environments for training and testing RL-based cyberdefense agents.

Findings

Framework enables modeling of complex, adaptive adversaries.

Supports training RL agents against poisoning and evasion attacks.

Facilitates development of robust, generalizable network defenders.

Abstract

Reinforcement learning (RL) has been demonstrated suitable to develop agents that play complex games with human-level performance. However, it is not understood how to effectively use RL to perform cybersecurity tasks. To develop such understanding, it is necessary to develop RL agents using simulation and emulation systems allowing researchers to model a broad class of realistic threats and network conditions. Demonstrating that a specific RL algorithm can be effective for defending a network under certain conditions may not necessarily give insight about the performance of the algorithm when the threats, network conditions, and security goals change. This paper introduces a novel approach for network environment design and a software framework to address the fundamental problem that network defense cannot be defined as a single game with a simple set of fixed rules. We show how our approach is necessary to facilitate the development of RL network defenders that are robust against attacks aimed at the agent's learning. Our framework enables the development and simulation of adversaries with sophisticated behavior that includes poisoning and evasion attacks on RL network defenders.