RAG-targeted Adversarial Attack on LLM-based Threat Detection and Mitigation Framework

TL;DR

This paper demonstrates how targeted data poisoning attacks can significantly weaken the effectiveness of LLM-based IoT threat detection systems by corrupting their knowledge base, exposing vulnerabilities in AI-driven cybersecurity frameworks.

Contribution

It introduces a novel adversarial attack method on RAG-based LLM threat detection, highlighting vulnerabilities and providing insights into improving robustness against data poisoning.

Findings

Small perturbations degrade model performance

Attack weakens link between traffic features and attack behavior

Mitigation recommendations become less specific and practical

Abstract

The rapid expansion of the Internet of Things (IoT) is reshaping communication and operational practices across industries, but it also broadens the attack surface and increases susceptibility to security breaches. Artificial Intelligence has become a valuable solution in securing IoT networks, with Large Language Models (LLMs) enabling automated attack behavior analysis and mitigation suggestion in Network Intrusion Detection Systems (NIDS). Despite advancements, the use of LLMs in such systems further expands the attack surface, putting entire networks at risk by introducing vulnerabilities such as prompt injection and data poisoning. In this work, we attack an LLM-based IoT attack analysis and mitigation framework to test its adversarial robustness. We construct an attack description dataset and use it in a targeted data poisoning attack that applies word-level, meaning-preserving perturbations to corrupt the Retrieval-Augmented Generation (RAG) knowledge base of the framework. We then compare pre-attack and post-attack mitigation responses from the target model, ChatGPT-5 Thinking, to measure the impact of the attack on model performance, using an established evaluation rubric designed for human experts and judge LLMs. Our results show that small perturbations degrade LLM performance by weakening the linkage between observed network traffic features and attack behavior, and by reducing the specificity and practicality of recommended mitigations for resource-constrained devices.