Compact Data Structures for Network Telemetry

TL;DR

This paper reviews how compact data structures and streaming algorithms are used to improve real-time network telemetry, addressing the challenges of limited device resources and the need for high-speed analysis.

Contribution

It provides a comprehensive review of recent advances in compact data structures for network telemetry and discusses future research directions.

Findings

Enhanced real-time traffic analysis capabilities

Trade-offs between accuracy and resource usage

Emerging research directions in the field

Abstract

Collecting and analyzing of network traffic data (network telemetry) plays a critical role in managing modern networks. Network administrators analyze their traffic to troubleshoot performance and reliability problems, and to detect and block cyberattacks. However, conventional traffic-measurement techniques offer limited visibility into network conditions and rely on offline analysis. Fortunately, network devices -- such as switches and network interface cards -- are increasingly programmable at the packet level, enabling flexible analysis of the traffic in place, as the packets fly by. However, to operate at high speed, these devices have limited memory and computational resources, leading to trade-offs between accuracy and overhead. In response, an exciting research area emerged, bringing ideas from compact data structures and streaming algorithms to bear on important networking telemetry applications and the unique characteristics of high-speed network devices. In this paper, we review the research on compact data structures for network telemetry and discuss promising directions for future research.