Learning Communication Between Heterogeneous Agents in Multi-Agent Reinforcement Learning for Autonomous Cyber Defence

TL;DR

This paper investigates how heterogeneous multi-agent reinforcement learning agents can communicate effectively to enhance cyber defense, demonstrating improved convergence speed and accuracy over existing algorithms in a simulated environment.

Contribution

It introduces the use of CommFormer for heterogeneous agents in cyber defense, showing significant performance improvements in a simulated network environment.

Findings

Agents with heterogeneous capabilities outperform others in convergence speed.

CommFormer agents reduce standard error by up to 38%.

Heterogeneous agents can be trained effectively for cyber security tasks.

Abstract

Reinforcement learning techniques are being explored as solutions to the threat of cyber attacks on enterprise networks. Recent research in the field of AI in cyber security has investigated the ability of homogeneous multi-agent reinforcement learning agents, capable of inter-agent communication, to respond to cyberattacks. This paper advances the study of learned communication in multi-agent systems by examining heterogeneous agent capabilities within a simulated network environment. To this end, we leverage CommFormer, a publicly available state-of-the-art communication algorithm, to train and evaluate agents within the Cyber Operations Research Gym (CybORG). Our results show that CommFormer agents with heterogeneous capabilities can outperform other algorithms deployed in the CybORG environment, by converging to an optimal policy up to four times faster while improving standard error by up 38%. The agents implemented in this project provide an additional avenue for exploration in the field of AI for cyber security, enabling further research involving realistic networks.