Adaptive NAD: Online and Self-adaptive Unsupervised Network Anomaly Detector

TL;DR

Adaptive NAD is an online, self-adaptive, and interpretable unsupervised anomaly detection framework for IoT security, capable of adapting to evolving threats and outperforming state-of-the-art methods in multiple datasets.

Contribution

It introduces a novel online learning scheme with an interpretable two-layer detection strategy for adaptive unsupervised anomaly detection in security applications.

Findings

Achieves over 5.4% SPAUC improvement on CIC-Darknet2020

Demonstrates 23.0% SPAUC improvement on CIC-DoHBrw-2020

Attains 3.2% SPAUC improvement on Edge-IIoTset

Abstract

The widespread usage of the Internet of Things (IoT) has raised the risks of cyber threats, thus developing Anomaly Detection Systems (ADSs) that can adapt to evolving or new attacks is critical. Previous studies primarily focused on offline unsupervised learning methods to safeguard ADSs, which is not applicable in practical real-world applications. Besides, most of them strongly rely on assumptions of known legitimates and fail to satisfy the interpretable requirements in security applications, creating barriers to the adoption in practice. In this paper, we design Adaptive NAD, a general framework to improve and interpret online unsupervised anomaly detection in security domains. An interpretable two-layer anomaly detection strategy is proposed to generate reliable high-confidence pseudo-labels. Then, an online learning scheme is introduced to update Adaptive NAD by a novel threshold calculation technique to adapt to new threats. Experimental results demonstrate that Adaptive NAD achieves more than 5.4%, 23.0%, and 3.2% improvements in SPAUC compared with state-of-the-art solutions on the CIC-Darknet2020, CIC-DoHBrw-2020, and Edge-IIoTset datasets, respectively. The code is released at https://github.com/MyLearnCodeSpace/Adaptive-NAD.