Learning to Communicate in Multi-Agent Reinforcement Learning for Autonomous Cyber Defence

TL;DR

This paper introduces a multi-agent reinforcement learning framework where autonomous cyber defense agents learn to communicate and coordinate effectively against cyber threats, improving decision-making in complex, partially observable environments.

Contribution

It presents a novel game-based training approach using Differentiable Inter Agent Learning for cyber defense, enabling agents to learn both tactical policies and minimal communication messages.

Findings

Agents develop human-like incident response tactics

Communication improves coordination and threat mitigation

Minimal communication messages are learned alongside defense policies

Abstract

Popular methods in cooperative Multi-Agent Reinforcement Learning with partially observable environments typically allow agents to act independently during execution, which may limit the coordinated effect of the trained policies. However, by sharing information such as known or suspected ongoing threats, effective communication can lead to improved decision-making in the cyber battle space. We propose a game design where defender agents learn to communicate and defend against imminent cyber threats by playing training games in the Cyber Operations Research Gym, using the Differentiable Inter Agent Learning algorithm adapted to the cyber operational environment. The tactical policies learned by these autonomous agents are akin to those of human experts during incident responses to avert cyber threats. In addition, the agents simultaneously learn minimal cost communication messages while learning their defence tactical policies.