Learning Appearance-motion Normality for Video Anomaly Detection

TL;DR

This paper introduces a novel two-stream auto-encoder framework with spatial-temporal memories and adversarial learning to improve video anomaly detection by independently learning appearance and motion normality and exploring their correlations.

Contribution

The proposed framework uniquely combines independent learning of appearance and motion normality with correlation exploration via adversarial learning, advancing video anomaly detection methods.

Findings

Achieves 98.1% AUC on UCSD Ped2 dataset.

Achieves 89.8% AUC on CUHK Avenue dataset.

Outperforms state-of-the-art methods in video anomaly detection.

Abstract

Video anomaly detection is a challenging task in the computer vision community. Most single task-based methods do not consider the independence of unique spatial and temporal patterns, while two-stream structures lack the exploration of the correlations. In this paper, we propose spatial-temporal memories augmented two-stream auto-encoder framework, which learns the appearance normality and motion normality independently and explores the correlations via adversarial learning. Specifically, we first design two proxy tasks to train the two-stream structure to extract appearance and motion features in isolation. Then, the prototypical features are recorded in the corresponding spatial and temporal memory pools. Finally, the encoding-decoding network performs adversarial learning with the discriminator to explore the correlations between spatial and temporal patterns. Experimental results show that our framework outperforms the state-of-the-art methods, achieving AUCs of 98.1% and 89.8% on UCSD Ped2 and CUHK Avenue datasets.