Enabling A Network AI Gym for Autonomous Cyber Agents

TL;DR

This paper introduces CyGIL, a unified training environment for autonomous cyber agents that combines emulation and simulation, enabling efficient training and transfer of RL agents to real network scenarios.

Contribution

The development of CyGIL, a hybrid environment that accelerates RL training and facilitates transferability of agents from simulation to real networks.

Findings

CyGIL-S trains agents in minutes, much faster than days in CyGIL-E.

Agents trained in CyGIL-S transfer effectively to CyGIL-E, demonstrating decision proficiency.

CyGIL enables offline RL and supports sim-to-real transfer for cyber agents.

Abstract

This work aims to enable autonomous agents for network cyber operations (CyOps) by applying reinforcement and deep reinforcement learning (RL/DRL). The required RL training environment is particularly challenging, as it must balance the need for high-fidelity, best achieved through real network emulation, with the need for running large numbers of training episodes, best achieved using simulation. A unified training environment, namely the Cyber Gym for Intelligent Learning (CyGIL) is developed where an emulated CyGIL-E automatically generates a simulated CyGIL-S. From preliminary experimental results, CyGIL-S is capable to train agents in minutes compared with the days required in CyGIL-E. The agents trained in CyGIL-S are transferrable directly to CyGIL-E showing full decision proficiency in the emulated "real" network. Enabling offline RL, the CyGIL solution presents a promising direction towards sim-to-real for leveraging RL agents in real-world cyber networks.