Taylor Polynomial Estimator for Estimating Frequency Moments

TL;DR

This paper introduces a new randomized algorithm for estimating the $p$th frequency moment in data streams, achieving improved space complexity bounds that match known lower bounds for various error parameters.

Contribution

The paper presents a novel algorithm that reduces space complexity for estimating frequency moments in data streams, matching existing lower bounds for certain error ranges.

Findings

Achieves space complexity $O(n^{1-2/p} \, \epsilon^{-2} + n^{1-2/p} \, \epsilon^{-4/p} \log n)$

Improves over previous bounds by Andoni et al.

Matches lower bounds by Li and Woodruff for specific error regimes.

Abstract

We present a randomized algorithm for estimating the $p$th moment $F_p$ of the frequency vector of a data stream in the general update (turnstile) model to within a multiplicative factor of $1 \pm \epsilon$, for $p > 2$, with high constant confidence. For $0 < \epsilon \le 1$, the algorithm uses space $O( n^{1-2/p} \epsilon^{-2} + n^{1-2/p} \epsilon^{-4/p} \log (n))$ words. This improves over the current bound of $O(n^{1-2/p} \epsilon^{-2-4/p} \log (n))$ words by Andoni et. al. in \cite{ako:arxiv10}. Our space upper bound matches the lower bound of Li and Woodruff \cite{liwood:random13} for $\epsilon = (\log (n))^{-\Omega(1)}$ and the lower bound of Andoni et. al. \cite{anpw:icalp13} for $\epsilon = \Omega(1)$.