Are Existing Out-Of-Distribution Techniques Suitable for Network Intrusion Detection?

TL;DR

This paper evaluates whether existing Out-Of-Distribution detection methods from other fields are effective for identifying unknown malicious network traffic, and finds that they can detect a significant portion of new attacks, especially with improved embeddings.

Contribution

It systematically assesses six OOD detection techniques in network intrusion detection and explores the impact of richer embedding spaces on detection performance.

Findings

Existing OOD detectors can identify a significant portion of new malicious traffic.

Enhanced embedding spaces improve detection effectiveness.

Combining detectors can achieve nearly 100% detection in tested scenarios.

Abstract

Machine learning (ML) has become increasingly popular in network intrusion detection. However, ML-based solutions always respond regardless of whether the input data reflects known patterns, a common issue across safety-critical applications. While several proposals exist for detecting Out-Of-Distribution (OOD) in other fields, it remains unclear whether these approaches can effectively identify new forms of intrusions for network security. New attacks, not necessarily affecting overall distributions, are not guaranteed to be clearly OOD as instead, images depicting new classes are in computer vision. In this work, we investigate whether existing OOD detectors from other fields allow the identification of unknown malicious traffic. We also explore whether more discriminative and semantically richer embedding spaces within models, such as those created with contrastive learning and multi-class tasks, benefit detection. Our investigation covers a set of six OOD techniques that employ different detection strategies. These techniques are applied to models trained in various ways and subsequently exposed to unknown malicious traffic from the same and different datasets (network environments). Our findings suggest that existing detectors can identify a consistent portion of new malicious traffic, and that improved embedding spaces enhance detection. We also demonstrate that simple combinations of certain detectors can identify almost 100% of malicious traffic in our tested scenarios.