ICE Buckets: Improved Counter Estimation for Network Measurement

TL;DR

This paper introduces ICE-Buckets, a novel algorithm that significantly improves the accuracy of large counter estimations in network measurement by optimizing estimation functions per bucket, enabling better traffic monitoring.

Contribution

The paper presents a closed form for the optimal estimation function and introduces ICE-Buckets, which enhance counter accuracy across all scales compared to prior estimators.

Findings

Achieves up to 57 times accuracy improvement on real Internet traces.

Provides a tighter upper bound on relative error.

Introduces a novel bucket-based estimation method.

Abstract

Measurement capabilities are essential for a variety of network applications, such as load balancing, routing, fairness and intrusion detection. These capabilities require large counter arrays in order to monitor the traffic of all network flows. While commodity SRAM memories are capable of operating at line speed, they are too small to accommodate large counter arrays. Previous works suggested estimators, which trade precision for reduced space. However, in order to accurately estimate the largest counter, these methods compromise the accuracy of the smaller counters. In this work, we present a closed form representation of the optimal estimation function. We then introduce Independent Counter Estimation Buckets (ICE-Buckets), a novel algorithm that improves estimation accuracy for all counters. This is achieved by separating the flows to buckets and configuring the optimal estimation function according to each bucket's counter scale. We prove a tighter upper bound on the relative error and demonstrate an accuracy improvement of up to 57 times on real Internet packet traces.