From Sands to Mansions: Towards Automated Cyberattack Emulation with Classical Planning and Large Language Models

TL;DR

This paper presents Aurora, an automated system that uses classical planning and large language models to generate realistic, causality-preserving cyberattack emulations from threat reports, improving security testing and benchmarking.

Contribution

It introduces a novel modular attack modeling framework and an automated emulation system combining symbolic planning and LLMs, creating a large, diverse attack dataset from real threat reports.

Findings

Aurora generates attack chains that are more diverse and realistic than existing methods.

The dataset from Aurora is 15 times larger than previous expert-crafted datasets.

Evaluation shows significant performance differences in intrusion detection systems on new datasets.

Abstract

Evolving attacker capabilities demand realistic and continuously updated cyberattack emulation for threat-informed defense and security benchmarking. Towards automated attack emulation, this paper defines modular attack actions and a linking model to organize and chain heterogeneous attack tools into causality-preserving cyberattacks. Building on this foundation, we introduce Aurora: an automated cyberattack emulation system powered by symbolic planning and large language models (LLMs). Aurora crafts actionable, causality-preserving attack chains tailored to Cyber Threat Intelligence (CTI) reports and target environments, and automatically executes these emulations. Using Aurora, we generated an extensive cyberattack emulation dataset from 250 attack reports, 15 times larger than the leading expert-crafted dataset. Our evaluation shows that Aurora significantly outperforms existing methods in creating actionable, diverse, and realistic attack chains. We release the dataset and use it to evaluate three state-of-the-art intrusion detection systems, whose performance differed notably from results on older datasets, highlighting the need for up-to-date, automated attack emulation.