Anomaly Detection at Scale: The Case for Deep Distributional Time Series Models

TL;DR

This paper proposes a novel deep distributional time series model for scalable anomaly detection in service monitoring, modeling probability distributions over data to improve accuracy and scalability.

Contribution

It introduces a new methodology that models time series of probability distributions, enabling scalable, streaming anomaly detection for millions of series.

Findings

Outperforms state-of-the-art on Yahoo Webscope datasets

Achieves up to 17% improvement over existing tools

Scales to millions of time series in real-time

Abstract

This paper introduces a new methodology for detecting anomalies in time series data, with a primary application to monitoring the health of (micro-) services and cloud resources. The main novelty in our approach is that instead of modeling time series consisting of real values or vectors of real values, we model time series of probability distributions over real values (or vectors). This extension to time series of probability distributions allows the technique to be applied to the common scenario where the data is generated by requests coming in to a service, which is then aggregated at a fixed temporal frequency. Our method is amenable to streaming anomaly detection and scales to monitoring for anomalies on millions of time series. We show the superior accuracy of our method on synthetic and public real-world data. On the Yahoo Webscope data set, we outperform the state of the art in 3 out of 4 data sets and we show that we outperform popular open-source anomaly detection tools by up to 17% average improvement for a real-world data set.