AutoBnB-RAG: Enhancing Multi-Agent Incident Response with Retrieval-Augmented Generation

TL;DR

AutoBnB-RAG enhances multi-agent incident response by integrating retrieval-augmented generation, enabling agents to access external knowledge and improve decision-making in simulated cybersecurity scenarios.

Contribution

This work introduces AutoBnB-RAG, a novel framework that incorporates retrieval-augmented generation into multi-agent incident response, improving reasoning and decision quality.

Findings

Retrieval augmentation improves decision success rates.

AutoBnB-RAG effectively reconstructs complex cyber attack scenarios.

Performance gains observed across various team structures.

Abstract

Incident response (IR) requires fast, coordinated, and well-informed decision-making to contain and mitigate cyber threats. While large language models (LLMs) have shown promise as autonomous agents in simulated IR settings, their reasoning is often limited by a lack of access to external knowledge. In this work, we present AutoBnB-RAG, an extension of the AutoBnB framework that incorporates retrieval-augmented generation (RAG) into multi-agent incident response simulations. Built on the Backdoors & Breaches (B&B) tabletop game environment, AutoBnB-RAG enables agents to issue retrieval queries and incorporate external evidence during collaborative investigations. We introduce two retrieval settings: one grounded in curated technical documentation (RAG-Wiki), and another using narrative-style incident reports (RAG-News). We evaluate performance across eight team structures, including newly introduced argumentative configurations designed to promote critical reasoning. To validate practical utility, we also simulate real-world cyber incidents based on public breach reports, demonstrating AutoBnB-RAG's ability to reconstruct complex multi-stage attacks. Our results show that retrieval augmentation improves decision quality and success rates across diverse organizational models. This work demonstrates the value of integrating retrieval mechanisms into LLM-based multi-agent systems for cybersecurity decision-making.