Modeling Behavioral Preferences of Cyber Adversaries Using Inverse Reinforcement Learning

TL;DR

This paper introduces a novel method using inverse reinforcement learning to model cyber adversaries' behavioral preferences from audit logs, providing a new dimension for threat attribution and understanding attacker invariants.

Contribution

It presents the first approach to automatically infer attacker preferences from forensic data, enhancing cyber threat modeling beyond tool and technique documentation.

Findings

Low-level forensic data can reveal attacker preferences.

Attacker preferences are invariant across different tools.

Inferred preferences can serve as behavioral signatures.

Abstract

This paper presents a holistic approach to attacker preference modeling from system-level audit logs using inverse reinforcement learning (IRL). Adversary modeling is an important capability in cybersecurity that lets defenders characterize behaviors of potential attackers, which enables attribution to known cyber adversary groups. Existing approaches rely on documenting an ever-evolving set of attacker tools and techniques to track known threat actors. Although attacks evolve constantly, attacker behavioral preferences are intrinsic and less volatile. Our approach learns the behavioral preferences of cyber adversaries from forensics data on their tools and techniques. We model the attacker as an expert decision-making agent with unknown behavioral preferences situated in a computer host. We leverage attack provenance graphs of audit logs to derive a state-action trajectory of the attack. We test our approach on open datasets of audit logs containing real attack data. Our results demonstrate for the first time that low-level forensics data can automatically reveal an adversary's subjective preferences, which serves as an additional dimension to modeling and documenting cyber adversaries. Attackers' preferences tend to be invariant despite their different tools and indicate predispositions that are inherent to the attacker. As such, these inferred preferences can potentially serve as unique behavioral signatures of attackers and improve threat attribution.