NetSentry: A Deep Learning Approach to Detecting Incipient Large-scale Network Attacks

TL;DR

NetSentry is a novel deep learning-based network intrusion detection system that leverages temporal correlations in early attack stages to detect large-scale network threats before they escalate, outperforming existing methods.

Contribution

The paper introduces NetSentry, the first NIDS using Bidirectional Asymmetric LSTM for early threat detection, and proposes a data augmentation technique to improve generalization across datasets.

Findings

F1 score improvements above 33% over state-of-the-art

Up to 3 times higher detection rates for XSS and web bruteforce

Data augmentation boosts F1 scores by over 35%

Abstract

Machine Learning (ML) techniques are increasingly adopted to tackle ever-evolving high-profile network attacks, including DDoS, botnet, and ransomware, due to their unique ability to extract complex patterns hidden in data streams. These approaches are however routinely validated with data collected in the same environment, and their performance degrades when deployed in different network topologies and/or applied on previously unseen traffic, as we uncover. This suggests malicious/benign behaviors are largely learned superficially and ML-based Network Intrusion Detection System (NIDS) need revisiting, to be effective in practice. In this paper we dive into the mechanics of large-scale network attacks, with a view to understanding how to use ML for Network Intrusion Detection (NID) in a principled way. We reveal that, although cyberattacks vary significantly in terms of payloads, vectors and targets, their early stages, which are critical to successful attack outcomes, share many similarities and exhibit important temporal correlations. Therefore, we treat NID as a time-sensitive task and propose NetSentry, perhaps the first of its kind NIDS that builds on Bidirectional Asymmetric LSTM (Bi-ALSTM), an original ensemble of sequential neural models, to detect network threats before they spread. We cross-evaluate NetSentry using two practical datasets, training on one and testing on the other, and demonstrate F1 score gains above 33% over the state-of-the-art, as well as up to 3 times higher rates of detecting attacks such as XSS and web bruteforce. Further, we put forward a novel data augmentation technique that boosts the generalization abilities of a broad range of supervised deep learning algorithms, leading to average F1 score gains above 35%.