REAL-IoT: Characterizing GNN Intrusion Detection Robustness under Practical Adversarial Attack

TL;DR

This paper introduces REAL-IoT, a comprehensive framework for evaluating the robustness of GNN-based IoT intrusion detection systems under realistic adversarial attacks and distribution drift, highlighting the importance of real-world testing.

Contribution

It presents a unified dataset, a physical IoT testbed, and explores LLM-based filtering to improve GNN robustness against realistic threats.

Findings

GNN models show significant performance drops under realistic attacks.

The unified dataset reveals generalization issues across different scenarios.

LLM filtering can mitigate adversarial impact effectively.

Abstract

Graph Neural Network (GNN)-based network intrusion detection systems (NIDS) are often evaluated on single datasets, limiting their ability to generalize under distribution drift. Furthermore, their adversarial robustness is typically assessed using synthetic perturbations that lack realism. This measurement gap leads to an overestimation of GNN-based NIDS resilience. To address the limitations, we propose \textbf{REAL-IoT}, a comprehensive framework for robustness evaluation of GNN-based NIDS in IoT environments. Our framework presents a methodology that creates a unified dataset from canonical datasets to assess generalization under drift. In addition, it features a novel intrusion dataset collected from a physical IoT testbed, which captures network traffic and attack scenarios under real-world settings. Furthermore, using REAL-IoT, we explore the usage of Large Language Models (LLMs) to analyze network data and mitigate the impact of adversarial examples by filtering suspicious flows. Our evaluations using REAL-IoT reveal performance drops in GNN models compared to results from standard benchmarks, quantifying their susceptibility to drift and realistic attacks. We also demonstrate the potential of LLM-based filtering to enhance robustness. These findings emphasize the necessity of realistic threat modeling and rigorous measurement practices for developing resilient IoT intrusion detection systems.