Memento: Making Sliding Windows Efficient for Heavy Hitters

TL;DR

Memento introduces efficient sliding window algorithms for real-time heavy hitter detection, significantly improving speed and accuracy over existing methods in cloud and network environments.

Contribution

The paper presents the first practical sliding window algorithms for heavy hitter detection, with extensive evaluations demonstrating substantial speed and accuracy improvements.

Findings

Single-device solutions are up to 273 times faster than existing techniques.

Network-wide detection reduces undetected flood requests by up to 37 times.

Memento is implemented as an open-source extension to HAProxy.

Abstract

Cloud operators require real-time identification of Heavy Hitters (HH) and Hierarchical Heavy Hitters (HHH) for applications such as load balancing, traffic engineering, and attack mitigation. However, existing techniques are slow in detecting new heavy hitters. In this paper, we make the case for identifying heavy hitters through \textit{sliding windows}. Sliding windows detect heavy hitters quicker and more accurately than current methods, but to date had no practical algorithms. Accordingly, we introduce, design and analyze the \textit{Memento} family of sliding window algorithms for the HH and HHH problems in the single-device and network-wide settings. Using extensive evaluations, we show that our single-device solutions attain similar accuracy and are by up to $273\times$ faster than existing window-based techniques. Furthermore, we exemplify our network-wide HHH detection capabilities on a realistic testbed. To that end, we implemented Memento as an open-source extension to the popular HAProxy cloud load-balancer. In our evaluations, using an HTTP flood by 50 subnets, our network-wide approach detected the new subnets faster, and reduced the number of undetected flood requests by up to $37\times$ compared to the alternatives.