A Study of Deep Learning for Network Traffic Data Forecasting

TL;DR

This paper explores deep learning for network traffic forecasting, introducing a large dataset, a new training scheme, and fine-grained prediction of bit rates, demonstrating the feasibility and challenges of advanced traffic prediction.

Contribution

It presents a novel large-scale dataset, a block-based training scheme, and fine-grained bit rate prediction, advancing network traffic forecasting methods.

Findings

Fine-grained prediction is feasible with deep learning.

Data enrichment improves forecasting accuracy.

Visualization techniques provide insights into complex traffic data.

Abstract

We present a study of deep learning applied to the domain of network traffic data forecasting. This is a very important ingredient for network traffic engineering, e.g., intelligent routing, which can optimize network performance, especially in large networks. In a nutshell, we wish to predict, in advance, the bit rate for a transmission, based on low-dimensional connection metadata ("flows") that is available whenever a communication is initiated. Our study has several genuinely new points: First, it is performed on a large dataset (~50 million flows), which requires a new training scheme that operates on successive blocks of data since the whole dataset is too large for in-memory processing. Additionally, we are the first to propose and perform a more fine-grained prediction that distinguishes between low, medium and high bit rates instead of just "mice" and "elephant" flows. Lastly, we apply state-of-the-art visualization and clustering techniques to flow data and show that visualizations are insightful despite the heterogeneous and non-metric nature of the data. We developed a processing pipeline to handle the highly non-trivial acquisition process and allow for proper data preprocessing to be able to apply DNNs to network traffic data. We conduct DNN hyper-parameter optimization as well as feature selection experiments, which clearly show that fine-grained network traffic forecasting is feasible, and that domain-dependent data enrichment and augmentation strategies can improve results. An outlook about the fundamental challenges presented by network traffic analysis (high data throughput, unbalanced and dynamic classes, changing statistics, outlier detection) concludes the article.