Adversarial Robustness for Machine Learning Cyber Defenses Using Log Data

TL;DR

This paper presents a framework to evaluate the adversarial robustness of machine learning-based cyber defenses analyzing log data, revealing vulnerabilities and how dropout regularization influences system resilience.

Contribution

It introduces a novel testing framework combining deep reinforcement learning and NLP techniques to assess adversarial robustness of log-based cyber defense systems.

Findings

Adversarial attacks can successfully compromise log-based ML defenses.

Higher dropout levels increase robustness against adversarial attacks.

90% dropout probability significantly enhances system resilience.

Abstract

There has been considerable and growing interest in applying machine learning for cyber defenses. One promising approach has been to apply natural language processing techniques to analyze logs data for suspicious behavior. A natural question arises to how robust these systems are to adversarial attacks. Defense against sophisticated attack is of particular concern for cyber defenses. In this paper, we develop a testing framework to evaluate adversarial robustness of machine learning cyber defenses, particularly those focused on log data. Our framework uses techniques from deep reinforcement learning and adversarial natural language processing. We validate our framework using a publicly available dataset and demonstrate that our adversarial attack does succeed against the target systems, revealing a potential vulnerability. We apply our framework to analyze the influence of different levels of dropout regularization and find that higher dropout levels increases robustness. Moreover 90% dropout probability exhibited the highest level of robustness by a significant margin, which suggests unusually high dropout may be necessary to properly protect against adversarial attacks.