RePAD: Real-time Proactive Anomaly Detection for Time Series

TL;DR

RePAD is a real-time proactive anomaly detection algorithm for streaming time series that predicts potential anomalies using LSTM and dynamically adjusts thresholds to provide early warnings without human intervention.

Contribution

It introduces RePAD, a novel LSTM-based method that detects and predicts anomalies proactively in streaming data without requiring domain knowledge or offline training.

Findings

Successfully detects anomalies early in real-time

Adapts to pattern changes with dynamic thresholds

Effective on datasets from Numenta Anomaly Benchmark

Abstract

During the past decade, many anomaly detection approaches have been introduced in different fields such as network monitoring, fraud detection, and intrusion detection. However, they require understanding of data pattern and often need a long off-line period to build a model or network for the target data. Providing real-time and proactive anomaly detection for streaming time series without human intervention and domain knowledge is highly valuable since it greatly reduces human effort and enables appropriate countermeasures to be undertaken before a disastrous damage, failure, or other harmful event occurs. However, this issue has not been well studied yet. To address it, this paper proposes RePAD, which is a Real-time Proactive Anomaly Detection algorithm for streaming time series based on Long Short-Term Memory (LSTM). RePAD utilizes short-term historic data points to predict and determine whether or not the upcoming data point is a sign that an anomaly is likely to happen in the near future. By dynamically adjusting the detection threshold over time, RePAD is able to tolerate minor pattern change in time series and detect anomalies either proactively or on time. Experiments based on two time series datasets collected from the Numenta Anomaly Benchmark demonstrate that RePAD is able to proactively detect anomalies and provide early warnings in real time without human intervention and domain knowledge.