High Speed Elephant Flow Detection Under Partial Information

TL;DR

This paper presents BubbleCache, a novel high-speed elephant flow detection algorithm that operates efficiently under partial information, reducing computational and memory costs significantly while maintaining high detection accuracy in real-world networks.

Contribution

The paper introduces a new theoretical framework and the BubbleCache algorithm for high-speed flow detection under uncertainty, balancing scalability and accuracy.

Findings

Reduces computational cost by a factor of 1000

Decreases memory requirements by a factor of 100

Achieves high probability detection of top flows in 100 Gbps networks

Abstract

In this paper we introduce a new framework to detect elephant flows at very high speed rates and under uncertainty. The framework provides exact mathematical formulas to compute the detection likelihood and introduces a new flow reconstruction lemma under partial information. These theoretical results lead to the design of BubbleCache, a new elephant flow detection algorithm designed to operate near the optimal tradeoff between computational scalability and accuracy by dynamically tracking the traffic's natural cutoff sampling rate. We demonstrate on a real world 100 Gbps network that the BubbleCache algorithm helps reduce the computational cost by a factor of 1000 and the memory requirements by a factor of 100 while detecting the top flows on the network with very high probability.