Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks

TL;DR

This paper introduces a novel unsupervised and semi-supervised anomaly detection method using LSTM and GRU neural networks combined with OC-SVM and SVDD, optimized jointly for high performance on variable length time series data.

Contribution

It is the first to jointly train LSTM/GRU networks with OC-SVM/SVDD using gradient and quadratic programming methods, extending anomaly detection to variable length sequences.

Findings

Significant performance improvements over conventional methods.

Effective joint training of neural networks with one-class classifiers.

Applicability to both time series and variable length data.

Abstract

We investigate anomaly detection in an unsupervised framework and introduce Long Short Term Memory (LSTM) neural network based algorithms. In particular, given variable length data sequences, we first pass these sequences through our LSTM based structure and obtain fixed length sequences. We then find a decision function for our anomaly detectors based on the One Class Support Vector Machines (OC-SVM) and Support Vector Data Description (SVDD) algorithms. As the first time in the literature, we jointly train and optimize the parameters of the LSTM architecture and the OC-SVM (or SVDD) algorithm using highly effective gradient and quadratic programming based training methods. To apply the gradient based training method, we modify the original objective criteria of the OC-SVM and SVDD algorithms, where we prove the convergence of the modified objective criteria to the original criteria. We also provide extensions of our unsupervised formulation to the semi-supervised and fully supervised frameworks. Thus, we obtain anomaly detection algorithms that can process variable length data sequences while providing high performance, especially for time series data. Our approach is generic so that we also apply this approach to the Gated Recurrent Unit (GRU) architecture by directly replacing our LSTM based structure with the GRU based structure. In our experiments, we illustrate significant performance gains achieved by our algorithms with respect to the conventional methods.