RLAD: Time Series Anomaly Detection through Reinforcement Learning and Active Learning

TL;DR

RLAD is a semi-supervised reinforcement learning and active learning-based algorithm for time series anomaly detection that adapts continuously, requires minimal manual tuning, and outperforms existing methods on multiple datasets.

Contribution

The paper introduces RLAD, a novel semi-supervised, reinforcement learning-based approach that does not assume underlying data mechanisms and outperforms state-of-the-art anomaly detection methods.

Findings

RLAD outperforms unsupervised methods by 1.58x in F1 score with 1% labels.

RLAD achieves up to 4.4x improvement with only 0.1% labels.

RLAD outperforms seven deep-learning algorithms on large datasets.

Abstract

We introduce a new semi-supervised, time series anomaly detection algorithm that uses deep reinforcement learning (DRL) and active learning to efficiently learn and adapt to anomalies in real-world time series data. Our model - called RLAD - makes no assumption about the underlying mechanism that produces the observation sequence and continuously adapts the detection model based on experience with anomalous patterns. In addition, it requires no manual tuning of parameters and outperforms all state-of-art methods we compare with, both unsupervised and semi-supervised, across several figures of merit. More specifically, we outperform the best unsupervised approach by a factor of 1.58 on the F1 score, with only 1% of labels and up to around 4.4x on another real-world dataset with only 0.1% of labels. We compare RLAD with seven deep-learning based algorithms across two common anomaly detection datasets with up to around 3M data points and between 0.28% to 2.65% anomalies.We outperform all of them across several important performance metrics.