Learning to Detect: A Data-driven Approach for Network Intrusion Detection

TL;DR

This paper presents a hierarchical, data-driven machine learning approach for network intrusion detection using NSL-KDD data, emphasizing unsupervised representation learning and oversampling techniques to improve detection accuracy.

Contribution

It introduces a hierarchical classification framework and explores the benefits of unsupervised learning and oversampling methods for intrusion detection.

Findings

Unsupervised representation learning improves binary intrusion detection.

Hierarchical classification enhances attack type identification.

Oversampling with SVM-SMOTE mitigates data imbalance issues.

Abstract

With massive data being generated daily and the ever-increasing interconnectivity of the world's Internet infrastructures, a machine learning based intrusion detection system (IDS) has become a vital component to protect our economic and national security. In this paper, we perform a comprehensive study on NSL-KDD, a network traffic dataset, by visualizing patterns and employing different learning-based models to detect cyber attacks. Unlike previous shallow learning and deep learning models that use the single learning model approach for intrusion detection, we adopt a hierarchy strategy, in which the intrusion and normal behavior are classified firstly, and then the specific types of attacks are classified. We demonstrate the advantage of the unsupervised representation learning model in binary intrusion detection tasks. Besides, we alleviate the data imbalance problem with SVM-SMOTE oversampling technique in 4-class classification and further demonstrate the effectiveness and the drawback of the oversampling mechanism with a deep neural network as a base model.