When LLMs Team Up: A Coordinated Attack Framework for Automated Cyber Intrusions

TL;DR

This paper introduces CAESAR, a multi-agent framework for LLMs that improves automated cyber intrusion tasks by modeling role boundaries, artifact provenance, and cost constraints, leading to better success rates.

Contribution

CAESAR decomposes intrusion workflows into typed roles with a coordination protocol, enhancing multi-stage LLM-agent performance and interpretability over single-agent approaches.

Findings

CAESAR outperforms single-agent baselines on 25 CTF tasks.

It reduces performance variance and improves success, especially in multi-step exploits.

Role structure aids in monitoring and transferring LLM-agent behavior.

Abstract

Automated intrusion-style workflows require LLM agents to reason over partial observations, tool outputs, and executable artifacts under bounded budgets. A single LLM instance often compresses evidence extraction, planning, execution, and validation into one context, which increases the risk of context drift and error propagation. Existing LLM-based multi-agent systems support general collaboration, but they do not explicitly model the role boundaries, artifact provenance, and cost constraints that characterize multi-stage intrusion workflows. This paper presents CAESAR, a coordinated multi-agent framework for controlled analysis of LLM-agent behavior in intrusion-style tasks. CAESAR decomposes the workflow into five typed roles and coordinates them through a bounded round protocol with a persistent knowledge base, a per-round workspace, validator-gated knowledge promotion, and capability-token write isolation. We evaluate CAESAR on 25 CTF tasks across five categories and four LLM backends. Compared with a single-agent baseline under matched budgets and tool access, CAESAR improves task success and reduces performance variance, with larger gains on tasks requiring multi-step exploit composition. A secondary simulated interactional-security study suggests that the role structure can transfer beyond code-native surfaces. The results indicate that role transitions, artifact provenance, and knowledge-promotion events provide useful structural signals for monitoring coordinated LLM-agent behavior beyond individual prompt and output inspection. The dataset, implementation, and evaluation logs are released at https://github.com/Xu-Qiu/CMAS.