Phase transition in count approximation by Count-Min sketch with conservative updates

TL;DR

This paper analyzes the Count-Min sketch with conservative updates, revealing a phase transition in counting accuracy depending on the number of distinct keys relative to the array size, with implications for hypergraph peelability.

Contribution

It establishes a phase transition in Count-Min sketch accuracy under conservative updates, linking it to hypergraph peelability and providing insights into distribution effects.

Findings

Below threshold, relative error is asymptotically o(1).

Above threshold, relative error is Theta(1).

Hypergraph peelability is key to maintaining low error.

Abstract

Count-Min sketch is a hash-based data structure to represent a dynamically changing associative array of counters. Here we analyse the counting version of Count-Min under a stronger update rule known as \textit{conservative update}, assuming the uniform distribution of input keys. We show that the accuracy of conservative update strategy undergoes a phase transition, depending on the number of distinct keys in the input as a fraction of the size of the Count-Min array. We prove that below the threshold, the relative error is asymptotically $o(1)$ (as opposed to the regular Count-Min strategy), whereas above the threshold, the relative error is $\Theta(1)$. The threshold corresponds to the peelability threshold of random $k$-uniform hypergraphs. We demonstrate that even for small number of keys, peelability of the underlying hypergraph is a crucial property to ensure the $o(1)$ error. Finally, we provide an experimental evidence that the phase transition does not extend to non-uniform distributions, in particular to the popular Zipf's distribution.