A Comparative Evaluation of Teacher-Guided Reinforcement Learning Techniques for Autonomous Cyber Operations

TL;DR

This paper evaluates four teacher-guided reinforcement learning techniques in autonomous cyber operations, showing that teacher integration enhances training efficiency and convergence speed in cybersecurity agents.

Contribution

It introduces and compares four teacher-guided RL methods applied to autonomous cyber operations, a novel approach in this domain.

Findings

Teacher-guided techniques improve early policy performance.

Teacher integration accelerates convergence in training.

Significant efficiency gains demonstrated in simulated environment.

Abstract

Autonomous Cyber Operations (ACO) rely on Reinforcement Learning (RL) to train agents to make effective decisions in the cybersecurity domain. However, existing ACO applications require agents to learn from scratch, leading to slow convergence and poor early-stage performance. While teacher-guided techniques have demonstrated promise in other domains, they have not yet been applied to ACO. In this study, we implement four distinct teacher-guided techniques in the simulated CybORG environment and conduct a comparative evaluation. Our results demonstrate that teacher integration can significantly improve training efficiency in terms of early policy performance and convergence speed, highlighting its potential benefits for autonomous cybersecurity.