CND-IDS: Continual Novelty Detection for Intrusion Detection Systems

TL;DR

CND-IDS is a continual novelty detection framework for intrusion detection systems that adapts to evolving data streams and detects new cyber attacks without relying on attack labels, significantly improving detection performance.

Contribution

It introduces a novel continual learning-based IDS framework combining feature extraction and PCA-based novelty detection, addressing data stream changes and label scarcity.

Findings

Achieves up to 6.1x F-score improvement over existing methods.

Achieves up to 6.5x better forward transfer compared to state-of-the-art unsupervised continual learning.

Effectively detects zero-day attacks in realistic intrusion datasets.

Abstract

Intrusion detection systems (IDS) play a crucial role in IoT and network security by monitoring system data and alerting to suspicious activities. Machine learning (ML) has emerged as a promising solution for IDS, offering highly accurate intrusion detection. However, ML-IDS solutions often overlook two critical aspects needed to build reliable systems: continually changing data streams and a lack of attack labels. Streaming network traffic and associated cyber attacks are continually changing, which can degrade the performance of deployed ML models. Labeling attack data, such as zero-day attacks, in real-world intrusion scenarios may not be feasible, making the use of ML solutions that do not rely on attack labels necessary. To address both these challenges, we propose CND-IDS, a continual novelty detection IDS framework which consists of (i) a learning-based feature extractor that continuously updates new feature representations of the system data, and (ii) a novelty detector that identifies new cyber attacks by leveraging principal component analysis (PCA) reconstruction. Our results on realistic intrusion datasets show that CND-IDS achieves up to 6.1x F-score improvement, and up to 6.5x improved forward transfer over the SOTA unsupervised continual learning algorithm. Our code will be released upon acceptance.