Catch Me If You Can: Improving Adversaries in Cyber-Security With Q-Learning Algorithms

TL;DR

This paper models cyberattackers using Q-Learning algorithms to enhance cybersecurity defenses, demonstrating that DoubleQ-Learning most effectively simulates attacker behavior with a 70% success rate.

Contribution

It introduces a Q-Learning based attacker model for cybersecurity, comparing variants to improve simulation accuracy for better defense calibration.

Findings

DoubleQ-Learning achieves 70% success in data exfiltration tasks.

Q-Learning variants effectively simulate attacker behavior.

Enhanced attacker models aid in developing more robust cybersecurity defenses.

Abstract

The ongoing rise in cyberattacks and the lack of skilled professionals in the cybersecurity domain to combat these attacks show the need for automated tools capable of detecting an attack with good performance. Attackers disguise their actions and launch attacks that consist of multiple actions, which are difficult to detect. Therefore, improving defensive tools requires their calibration against a well-trained attacker. In this work, we propose a model of an attacking agent and environment and evaluate its performance using basic Q-Learning, Naive Q-learning, and DoubleQ-Learning, all of which are variants of Q-Learning. The attacking agent is trained with the goal of exfiltrating data whereby all the hosts in the network have a non-zero detection probability. Results show that the DoubleQ-Learning agent has the best overall performance rate by successfully achieving the goal in $70\%$ of the interactions.