Adaptive algorithms for identifying large flows in IP traffic

TL;DR

This paper introduces an adaptive, online Bloom filter-based algorithm for identifying large IP flows and detecting anomalies like SYN floods, outperforming traditional refresh schemes in real traffic scenarios.

Contribution

The paper presents a novel adaptive scheme that improves large flow detection accuracy and anomaly detection speed in IP traffic analysis.

Findings

High detection ratio of long flows over extended periods

Anomaly detection within less than one minute

Effective identification of targeted destinations

Abstract

We propose in this paper an on-line algorithm based on Bloom filters for identifying large flows in IP traffic (a.k.a. elephants). Because of the large number of small flows, hash tables of these algorithms have to be regularly refreshed. Recognizing that the periodic erasure scheme usually used in the technical literature turns out to be quite inefficient when using real traffic traces over a long period of time, we introduce a simple adaptive scheme that closely follows the variations of traffic. When tested against real traffic traces, the proposed on-line algorithm performs well in the sense that the detection ratio of long flows by the algorithm over a long time period is quite high. Beyond the identification of elephants, this same class of algorithms is applied to the closely related problem of detection of anomalies in IP traffic, e.g., SYN flood due for instance to attacks. An algorithm for detecting SYN and volume flood anomalies in Internet traffic is designed. Experiments show that an anomaly is detected in less than one minute and the targeted destinations are identified at the same time.