Video Anomaly Detection and Localization via Gaussian Mixture Fully Convolutional Variational Autoencoder

TL;DR

This paper introduces a novel deep learning framework using Gaussian Mixture Variational Autoencoders and fully convolutional networks for effective video anomaly detection and localization, leveraging only normal samples for training.

Contribution

It proposes a partially supervised method that models normal data with Gaussian mixtures and detects anomalies via sample energy, combining appearance and motion analysis.

Findings

Outperforms state-of-the-art on UCSD and Avenue datasets.

Effectively models normal patterns with Gaussian Mixture Variational Autoencoders.

Accurately localizes anomalies in video frames.

Abstract

We present a novel end-to-end partially supervised deep learning approach for video anomaly detection and localization using only normal samples. The insight that motivates this study is that the normal samples can be associated with at least one Gaussian component of a Gaussian Mixture Model (GMM), while anomalies either do not belong to any Gaussian component. The method is based on Gaussian Mixture Variational Autoencoder, which can learn feature representations of the normal samples as a Gaussian Mixture Model trained using deep learning. A Fully Convolutional Network (FCN) that does not contain a fully-connected layer is employed for the encoder-decoder structure to preserve relative spatial coordinates between the input image and the output feature map. Based on the joint probabilities of each of the Gaussian mixture components, we introduce a sample energy based method to score the anomaly of image test patches. A two-stream network framework is employed to combine the appearance and motion anomalies, using RGB frames for the former and dynamic flow images, for the latter. We test our approach on two popular benchmarks (UCSD Dataset and Avenue Dataset). The experimental results verify the superiority of our method compared to the state of the arts.