Proof: Accelerating Approximate Aggregation Queries with Expensive Predicates

TL;DR

This paper introduces ABae, a stratified sampling method with proxy models for faster approximate aggregation queries with expensive predicates, and provides a theoretical analysis of its mean squared error decay rate.

Contribution

It offers a theoretical analysis of ABae, showing its MSE decay rate and demonstrating it can match optimal stratified sampling performance with proper sample allocation.

Findings

MSE of ABae decays at rate O(N^{-1}) with proper sample allocation

ABae achieves near-optimal performance compared to known-stratum algorithms

Theoretical bounds validate ABae's efficiency for approximate aggregation

Abstract

Given a dataset $\mathcal{D}$, we are interested in computing the mean of a subset of $\mathcal{D}$ which matches a predicate. ABae leverages stratified sampling and proxy models to efficiently compute this statistic given a sampling budget $N$. In this document, we theoretically analyze ABae and show that the MSE of the estimate decays at rate $O(N_1^{-1} + N_2^{-1} + N_1^{1/2}N_2^{-3/2})$, where $N=K \cdot N_1+N_2$ for some integer constant $K$ and $K \cdot N_1$ and $N_2$ represent the number of samples used in Stage 1 and Stage 2 of ABae respectively. Hence, if a constant fraction of the total sample budget $N$ is allocated to each stage, we will achieve a mean squared error of $O(N^{-1})$ which matches the rate of mean squared error of the optimal stratified sampling algorithm given a priori knowledge of the predicate positive rate and standard deviation per stratum.