Estimation of Multiple Quantiles in Dynamically Varying Data Streams

TL;DR

This paper introduces an efficient method for real-time estimation of multiple quantiles in data streams with changing distributions, ensuring monotonicity and outperforming existing techniques.

Contribution

The paper proposes a novel incremental approach for simultaneously estimating multiple quantiles in dynamic data streams with guaranteed monotonicity.

Findings

Method efficiently tracks multiple quantiles in real-time.

Outperforms state-of-the-art quantile estimation methods.

Maintains monotonicity of quantile estimates throughout.

Abstract

In this paper we consider the problem of estimating quantiles when data are received sequentially (data stream). For real life data streams, the distribution of the data typically varies with time making estimation of quantiles challenging. We present a method that simultaneously maintain estimates of multiple quantiles of the data stream distribution. The method is based on making incremental updates of the quantile estimates every time a new sample from the data stream is received. The method is memory and computationally efficient since it only stores one value for each quantile estimate and only performs one operation per quantile estimate when a new sample is received from the data stream. The estimates are realistic in the sense that the monotone property of quantiles is satisfied in every iteration. Experiments show that the method efficiently tracks multiple quantiles and outperforms state of the art methods.