Learning Deep Representations of Appearance and Motion for Anomalous Event Detection

TL;DR

This paper introduces an unsupervised deep learning framework called AMDN for detecting anomalous events in videos by automatically learning appearance and motion features through a novel fusion approach.

Contribution

It proposes a deep neural network-based method that combines appearance and motion features using a double fusion framework for improved anomaly detection.

Findings

Achieves competitive results on public datasets

Effectively combines appearance and motion information

Outperforms some existing methods in accuracy

Abstract

We present a novel unsupervised deep learning framework for anomalous event detection in complex video scenes. While most existing works merely use hand-crafted appearance and motion features, we propose Appearance and Motion DeepNet (AMDN) which utilizes deep neural networks to automatically learn feature representations. To exploit the complementary information of both appearance and motion patterns, we introduce a novel double fusion framework, combining both the benefits of traditional early fusion and late fusion strategies. Specifically, stacked denoising autoencoders are proposed to separately learn both appearance and motion features as well as a joint representation (early fusion). Based on the learned representations, multiple one-class SVM models are used to predict the anomaly scores of each input, which are then integrated with a late fusion strategy for final anomaly detection. We evaluate the proposed method on two publicly available video surveillance datasets, showing competitive performance with respect to state of the art approaches.