Count-Min-Log sketch: Approximately counting with approximate counters

TL;DR

The paper introduces Count-Min-Log sketch, an improved variant of Count-Min Sketch that uses logarithmic counters to enhance accuracy in low-frequency event counting within fixed memory constraints.

Contribution

It proposes a novel Count-Min-Log sketch that replaces linear counters with logarithmic counters to reduce relative error for low-frequency items.

Findings

Improved accuracy for low-frequency event counts.

Maintains constant memory footprint while reducing errors.

Outperforms traditional Count-Min Sketch in experiments.

Abstract

Count-Min Sketch is a widely adopted algorithm for approximate event counting in large scale processing. However, the original version of the Count-Min-Sketch (CMS) suffers of some deficiences, especially if one is interested by the low-frequency items, such as in text-mining related tasks. Several variants of CMS have been proposed to compensate for the high relative error for low-frequency events, but the proposed solutions tend to correct the errors instead of preventing them. In this paper, we propose the Count-Min-Log sketch, which uses logarithm-based, approximate counters instead of linear counters to improve the average relative error of CMS at constant memory footprint.