Sequence Aggregation Rules for Anomaly Detection in Computer Network Traffic

TL;DR

This paper compares sequence aggregation rules and models, including LSTM and frequency-based approaches, for unsupervised anomaly detection in network traffic, finding simple frequency methods often outperform complex neural networks.

Contribution

It introduces and evaluates five sequence aggregation rules and compares LSTM-based models with a simple frequency-based model for anomaly detection in network traffic.

Findings

Frequency-based model performs as well or better than LSTM models

LSTM models show limited advantage over simple frequency methods

Evaluation conducted on the CICIDS2017 dataset

Abstract

We evaluate methods for applying unsupervised anomaly detection to cybersecurity applications on computer network traffic data, or flow. We borrow from the natural language processing literature and conceptualize flow as a sort of "language" spoken between machines. Five sequence aggregation rules are evaluated for their efficacy in flagging multiple attack types in a labeled flow dataset, CICIDS2017. For sequence modeling, we rely on long short-term memory (LSTM) recurrent neural networks (RNN). Additionally, a simple frequency-based model is described and its performance with respect to attack detection is compared to the LSTM models. We conclude that the frequency-based model tends to perform as well as or better than the LSTM models for the tasks at hand, with a few notable exceptions.