AutoFlow: An Autoencoder-based Approach for IP Flow Record Compression with Minimal Impact on Traffic Classification

TL;DR

This paper introduces AutoFlow, an autoencoder-based method for compressing IP flow records that significantly reduces data size while preserving the accuracy of traffic classification, enabling more efficient network monitoring.

Contribution

The paper proposes a novel deep learning approach using autoencoders for compressing IP flow data with minimal impact on classification accuracy, outperforming traditional methods.

Findings

Achieves 1.313x data size reduction

Maintains 99.27% accuracy in traffic classification

Enhances storage and processing efficiency

Abstract

Network monitoring generates massive volumes of IP flow records, posing significant challenges for storage and analysis. This paper presents a novel deep learning-based approach to compressing these records using autoencoders, enabling direct analysis of compressed data without requiring decompression. Unlike traditional compression methods, our approach reduces data volume while retaining the utility of compressed data for downstream analysis tasks, including distinguishing modern application protocols and encrypted traffic from popular services. Through extensive experiments on a real-world network traffic dataset, we demonstrate that our autoencoder-based compression achieves a 1.313x reduction in data size while maintaining 99.27% accuracy in a multi-class traffic classification task, compared to 99.77% accuracy with uncompressed data. This marginal decrease in performance is offset by substantial gains in storage and processing efficiency. The implications of this work extend to more efficient network monitoring and scalable, real-time network management solutions.