Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learning

TL;DR

This paper demonstrates that cooperative multi-agent reinforcement learning can effectively develop cyber defence tactics in simulated environments, outperforming heuristic methods against varied cyber threats.

Contribution

It introduces a comparison of MARL approaches for cyber defence, showing their ability to learn effective tactics against attacker scenarios.

Findings

MARL outperforms heuristic defenders in simulated cyber defence tasks.

Both value-based and CTDE MARL methods are effective.

Cooperative MARL can adapt to different attack strategies.

Abstract

Recent advancements in deep learning techniques have opened new possibilities for designing solutions for autonomous cyber defence. Teams of intelligent agents in computer network defence roles may reveal promising avenues to safeguard cyber and kinetic assets. In a simulated game environment, agents are evaluated on their ability to jointly mitigate attacker activity in host-based defence scenarios. Defender systems are evaluated against heuristic attackers with the goals of compromising network confidentiality, integrity, and availability. Value-based Independent Learning and Centralized Training Decentralized Execution (CTDE) cooperative Multi-Agent Reinforcement Learning (MARL) methods are compared revealing that both approaches outperform a simple multi-agent heuristic defender. This work demonstrates the ability of cooperative MARL to learn effective cyber defence tactics against varied threats.