Fast Detection of Outliers in Data Streams with the $Q_n$ Estimator

TL;DR

The paper introduces FQN, a fast and efficient algorithm for detecting outliers in data streams using the $Q_n$ estimator, outperforming existing methods in speed, space, and distribution independence.

Contribution

It presents a novel online $Q_n$ estimator algorithm for real-time outlier detection in data streams, improving speed and resource usage over prior methods.

Findings

FQN is faster than existing algorithms.

FQN's complexity is independent of data distribution.

FQN requires less memory than competitors.

Abstract

We present FQN (Fast $Q_n$), a novel algorithm for fast detection of outliers in data streams. The algorithm works in the sliding window model, checking if an item is an outlier by cleverly computing the $Q_n$ scale estimator in the current window. We thoroughly compare our algorithm for online $Q_n$ with the state of the art competing algorithm by Nunkesser et al, and show that FQN (i) is faster, (ii) its computational complexity does not depend on the input distribution and (iii) it requires less space. Extensive experimental results on synthetic datasets confirm the validity of our approach.