Quantifying Catastrophic Forgetting in IoT Intrusion Detection Systems

TL;DR

This paper addresses the challenge of catastrophic forgetting in IoT intrusion detection systems by formulating it as a continual learning problem and benchmarking various strategies to improve adaptability and robustness.

Contribution

It introduces a domain continual learning framework for IDS and systematically evaluates multiple approaches on a multi-attack IoT dataset.

Findings

Replay-based methods perform best overall.

Synaptic Intelligence achieves near-zero forgetting.

Continual learning balances plasticity and stability in resource-constrained IoT environments.

Abstract

Distribution shifts in attack patterns within RPL-based IoT networks pose a critical threat to the reliability and security of large-scale connected systems. Intrusion Detection Systems (IDS) trained on static datasets often fail to generalize to unseen threats and suffer from catastrophic forgetting when updated with new attacks. Ensuring continual adaptability of IDS is therefore essential for maintaining robust IoT network defence. In this focused study, we formulate intrusion detection as a domain continual learning problem and propose a method-agnostic IDS framework that can integrate diverse continual learning strategies. We systematically benchmark five representative approaches across multiple domain-ordering sequences using a comprehensive multi-attack dataset comprising 48 domains. Results show that continual learning mitigates catastrophic forgetting while maintaining a balance between plasticity, stability, and efficiency, a crucial aspect for resource-constrained IoT environments. Among the methods, Replay-based approaches achieve the best overall performance, while Synaptic Intelligence (SI) delivers near-zero forgetting with high training efficiency, demonstrating strong potential for stable and sustainable IDS deployment in dynamic IoT networks.