Real-time Anomaly Detection for Multivariate Data Streams

TL;DR

This paper introduces a real-time, unsupervised multivariate anomaly detection algorithm for data streams that is resilient to various types of distributional shifts and concept drifts, using a novel PEWMA-based approach.

Contribution

The paper proposes a new PEWMA-based anomaly detection method that handles concept drifts and shifts in real-time without requiring labeled data.

Findings

Performs well in real-time anomaly detection

Resilient to abrupt and gradual distributional shifts

Operates effectively in unsupervised settings

Abstract

We present a real-time multivariate anomaly detection algorithm for data streams based on the Probabilistic Exponentially Weighted Moving Average (PEWMA). Our formulation is resilient to (abrupt transient, abrupt distributional, and gradual distributional) shifts in the data. The novel anomaly detection routines utilize an incremental online algorithm to handle streams. Furthermore, our proposed anomaly detection algorithm works in an unsupervised manner eliminating the need for labeled examples. Our algorithm performs well and is resilient in the face of concept drifts.