Unsupervised Anomaly Detection on Temporal Multiway Data

TL;DR

This paper explores unsupervised anomaly detection in temporal multiway data using matrix-native RNNs, revealing new insights into model behaviors and effective strategies like encoding-predicting.

Contribution

It introduces and evaluates strategies for anomaly detection on two-way temporal data with matrix RNNs, highlighting the effectiveness of encoding-predicting and revealing phenomena like noise compression.

Findings

Matrix LSTM can compress noisy data nearly perfectly.

Encoding-predicting strategy outperforms others in experiments.

Matrix models handle very long sequences effectively.

Abstract

Temporal anomaly detection looks for irregularities over space-time. Unsupervised temporal models employed thus far typically work on sequences of feature vectors, and much less on temporal multiway data. We focus our investigation on two-way data, in which a data matrix is observed at each time step. Leveraging recent advances in matrix-native recurrent neural networks, we investigated strategies for data arrangement and unsupervised training for temporal multiway anomaly detection. These include compressing-decompressing, encoding-predicting, and temporal data differencing. We conducted a comprehensive suite of experiments to evaluate model behaviors under various settings on synthetic data, moving digits, and ECG recordings. We found interesting phenomena not previously reported. These include the capacity of the compact matrix LSTM to compress noisy data near perfectly, making the strategy of compressing-decompressing data ill-suited for anomaly detection under the noise. Also, long sequence of vectors can be addressed directly by matrix models that allow very long context and multiple step prediction. Overall, the encoding-predicting strategy works very well for the matrix LSTMs in the conducted experiments, thanks to its compactness and better fit to the data dynamics.