On Efficient Range-Summability of Ideally IID Random Variables in Two or Higher Dimensions

TL;DR

This paper introduces the first polylogarithmic time solutions for efficient range-summability of IID Gaussian or Poisson random variables in higher dimensions, advancing database query and data stream processing.

Contribution

It presents a novel framework for $d$D-ERS with polylogarithmic complexity and develops a new $k$-wise independence theory, enabling efficient and provably independent solutions in higher dimensions.

Findings

First polylogarithmic time solutions for $d$D-ERS with Gaussian or Poisson RVs.

Development of a new $k$-wise independence theory for high efficiency and strong guarantees.

Generalization of 1D-ERS solutions to higher dimensions under certain conditions.

Abstract

$d$-dimensional (for $d>1$) efficient range-summability ($d$D-ERS) of random variables (RVs) is a fundamental algorithmic problem that has applications to two important families of database problems, namely, fast approximate wavelet tracking (FAWT) on data streams and approximately answering range-sum queries over a data cube. Whether there are efficient solutions to the $d$D-ERS problem, or to the latter database problem, have been two long-standing open problems. Both are solved in this work. Specifically, we propose a novel solution framework to $d$D-ERS on RVs that have Gaussian or Poisson distribution. Our $d$D-ERS solutions are the first ones that have polylogarithmic time complexities. Furthermore, we develop a novel $k$-wise independence theory that allows our $d$D-ERS solutions to have both high computational efficiencies and strong provable independence guarantees. Finally, we show that under a sufficient and likely necessary condition, certain existing solutions for 1D-ERS can be generalized to higher dimensions.