Efficient Summing over Sliding Windows

TL;DR

This paper develops optimal algorithms for maintaining approximate sums over sliding windows in data streams, achieving tight memory bounds and constant-time processing for both binary and integer streams.

Contribution

It introduces memory-efficient algorithms with tight bounds for summing over sliding windows, extending to randomized algorithms and various error regimes.

Findings

Memory bounds are tight for counting 1's in sliding windows.

Algorithms operate in O(1) worst-case time per update and query.

New algorithms handle sums of integers with different error tolerances.

Abstract

This paper considers the problem of maintaining statistic aggregates over the last W elements of a data stream. First, the problem of counting the number of 1's in the last W bits of a binary stream is considered. A lower bound of {\Omega}(1/{\epsilon} + log W) memory bits for W{\epsilon}-additive approximations is derived. This is followed by an algorithm whose memory consumption is O(1/{\epsilon} + log W) bits, indicating that the algorithm is optimal and that the bound is tight. Next, the more general problem of maintaining a sum of the last W integers, each in the range of {0,1,...,R}, is addressed. The paper shows that approximating the sum within an additive error of RW{\epsilon} can also be done using {\Theta}(1/{\epsilon} + log W) bits for {\epsilon}={\Omega}(1/W). For {\epsilon}=o(1/W), we present a succinct algorithm which uses B(1 + o(1)) bits, where B={\Theta}(Wlog(1/W{\epsilon})) is the derived lower bound. We show that all lower bounds generalize to randomized algorithms as well. All algorithms process new elements and answer queries in O(1) worst-case time.