Heavy-Hitter Detection Entirely in the Data Plane

TL;DR

HashPipe is a novel data-plane algorithm implemented in P4 that efficiently detects heavy hitter flows using limited memory, suitable for high-speed network environments.

Contribution

It introduces HashPipe, a pipeline-based heavy hitter detection method optimized for programmable data planes with constrained memory and processing speed.

Findings

Identifies 95% of the heaviest flows in ISP traces

Uses less than 80KB of memory for detection

Effective in high-speed network environments

Abstract

Identifying the "heavy hitter" flows or flows with large traffic volumes in the data plane is important for several applications e.g., flow-size aware routing, DoS detection, and traffic engineering. However, measurement in the data plane is constrained by the need for line-rate processing (at 10-100Gb/s) and limited memory in switching hardware. We propose HashPipe, a heavy hitter detection algorithm using emerging programmable data planes. HashPipe implements a pipeline of hash tables which retain counters for heavy flows while evicting lighter flows over time. We prototype HashPipe in P4 and evaluate it with packet traces from an ISP backbone link and a data center. On the ISP trace (which contains over 400,000 flows), we find that HashPipe identifies 95% of the 300 heaviest flows with less than 80KB of memory.