On Autonomous Agents in a Cyber Defence Environment

TL;DR

This paper evaluates autonomous agents in cyber defence scenarios using the CAGE environment, highlighting hierarchical deep reinforcement learning as the most effective approach for defending networks against attacks.

Contribution

It provides a detailed analysis of challenge approaches and identifies hierarchical DRL as the most capable algorithm for autonomous cyber defence.

Findings

Hierarchical DRL outperformed other algorithms in the challenge.

Different algorithms produced diverse defensive strategies.

Defensive strategies varied based on offensive tactics.

Abstract

Autonomous Cyber Defence is required to respond to high-tempo cyber-attacks. To facilitate the research in this challenging area, we explore the utility of the autonomous cyber operation environments presented as part of the Cyber Autonomy Gym for Experimentation (CAGE) Challenges, with a specific focus on CAGE Challenge 2. CAGE Challenge 2 required a defensive Blue agent to defend a network from an attacking Red agent. We provide a detailed description of the this challenge and describe the approaches taken by challenge participants. From the submitted agents, we identify four classes of algorithms, namely, Single- Agent Deep Reinforcement Learning (DRL), Hierarchical DRL, Ensembles, and Non-DRL approaches. Of these classes, we found that the hierarchical DRL approach was the most capable of learning an effective cyber defensive strategy. Our analysis of the agent policies identified that different algorithms within the same class produced diverse strategies and that the strategy used by the defensive Blue agent varied depending on the strategy used by the offensive Red agent. We conclude that DRL algorithms are a suitable candidate for autonomous cyber defence applications.