Constrained Network Adversarial Attacks: Validity, Robustness, and Transferability

TL;DR

This paper highlights the importance of domain constraints in adversarial attacks on IoT network intrusion detection systems, revealing that many attacks are invalid and emphasizing the need for realistic evaluation methods.

Contribution

It identifies the flaw of ignoring domain constraints in adversarial attack generation and demonstrates the impact on attack validity and transferability in IoT security models.

Findings

Up to 80.3% of adversarial examples are invalid due to constraint violations.

Simpler surrogate models like MLP produce more valid adversarial examples.

Transferability of adversarial severity varies with model complexity and constraints.

Abstract

While machine learning has significantly advanced Network Intrusion Detection Systems (NIDS), particularly within IoT environments where devices generate large volumes of data and are increasingly susceptible to cyber threats, these models remain vulnerable to adversarial attacks. Our research reveals a critical flaw in existing adversarial attack methodologies: the frequent violation of domain-specific constraints, such as numerical and categorical limits, inherent to IoT and network traffic. This leads to up to 80.3% of adversarial examples being invalid, significantly overstating real-world vulnerabilities. These invalid examples, though effective in fooling models, do not represent feasible attacks within practical IoT deployments. Consequently, relying on these results can mislead resource allocation for defense, inflating the perceived susceptibility of IoT-enabled NIDS models to adversarial manipulation. Furthermore, we demonstrate that simpler surrogate models like Multi-Layer Perceptron (MLP) generate more valid adversarial examples compared to complex architectures such as CNNs and LSTMs. Using the MLP as a surrogate, we analyze the transferability of adversarial severity to other ML/DL models commonly used in IoT contexts. This work underscores the importance of considering both domain constraints and model architecture when evaluating and designing robust ML/DL models for security-critical IoT and network applications.