A Sampling Algebra for Aggregate Estimation

TL;DR

This paper introduces a theoretical framework based on generalized uniform sampling (GUS) that enables accurate aggregate estimation and confidence interval derivation in database systems, accommodating various sampling techniques.

Contribution

It presents a novel algebraic approach to analyze GUS sampling methods, allowing operators to commute with selection and join, improving estimate accuracy and confidence interval calculation.

Findings

GUS sampling operators commute with selection and join.

The framework enables derivation of confidence intervals for estimates.

Extensive examples demonstrate practical applicability.

Abstract

As of 2005, sampling has been incorporated in all major database systems. While efficient sampling techniques are realizable, determining the accuracy of an estimate obtained from the sample is still an unresolved problem. In this paper, we present a theoretical framework that allows an elegant treatment of the problem. We base our work on generalized uniform sampling (GUS), a class of sampling methods that subsumes a wide variety of sampling techniques. We introduce a key notion of equivalence that allows GUS sampling operators to commute with selection and join, and derivation of confidence intervals. We illustrate the theory through extensive examples and give indications on how to use it to provide meaningful estimations in database systems.