Detecting Abnormal Traffic in Large-Scale Networks

TL;DR

This paper reviews machine learning methods for detecting network attacks and introduces a new deep learning framework using LSTM autoencoders that improves detection accuracy in large-scale networks.

Contribution

It presents a novel LSTM autoencoder-based deep learning framework for more accurate detection of malicious network traffic, addressing challenges of imbalanced datasets.

Findings

Significant improvement over benchmark methods in attack detection accuracy

Effective detection of minority attack classes in imbalanced datasets

Validated on publicly available intrusion detection datasets

Abstract

With the rapid technological advancements, organizations need to rapidly scale up their information technology (IT) infrastructure viz. hardware, software, and services, at a low cost. However, the dynamic growth in the network services and applications creates security vulnerabilities and new risks that can be exploited by various attacks. For example, User to Root (U2R) and Remote to Local (R2L) attack categories can cause a significant damage and paralyze the entire network system. Such attacks are not easy to detect due to the high degree of similarity to normal traffic. While network anomaly detection systems are being widely used to classify and detect malicious traffic, there are many challenges to discover and identify the minority attacks in imbalanced datasets. In this paper, we provide a detailed and systematic analysis of the existing Machine Learning (ML) approaches that can tackle most of these attacks. Furthermore, we propose a Deep Learning (DL) based framework using Long Short Term Memory (LSTM) autoencoder that can accurately detect malicious traffics in network traffic. We perform our experiments in a publicly available dataset of Intrusion Detection Systems (IDSs). We obtain a significant improvement in attack detection, as compared to other benchmarking methods. Hence, our method provides great confidence in securing these networks from malicious traffic.