Convolutional Transformer based Dual Discriminator Generative Adversarial Networks for Video Anomaly Detection

TL;DR

This paper introduces CT-D2GAN, a novel unsupervised video anomaly detection method using convolutional transformers and dual discriminators to effectively model spatio-temporal patterns and detect anomalies in surveillance videos.

Contribution

The paper proposes a convolutional transformer architecture combined with dual discriminators for improved unsupervised video anomaly detection, explicitly capturing local and global temporal coherence.

Findings

Effective anomaly detection on UCSD Ped2, CUHK Avenue, Shanghai Tech datasets.

Outperforms existing methods in accuracy and robustness.

Demonstrates the importance of local and global spatio-temporal modeling.

Abstract

Detecting abnormal activities in real-world surveillance videos is an important yet challenging task as the prior knowledge about video anomalies is usually limited or unavailable. Despite that many approaches have been developed to resolve this problem, few of them can capture the normal spatio-temporal patterns effectively and efficiently. Moreover, existing works seldom explicitly consider the local consistency at frame level and global coherence of temporal dynamics in video sequences. To this end, we propose Convolutional Transformer based Dual Discriminator Generative Adversarial Networks (CT-D2GAN) to perform unsupervised video anomaly detection. Specifically, we first present a convolutional transformer to perform future frame prediction. It contains three key components, i.e., a convolutional encoder to capture the spatial information of the input video clips, a temporal self-attention module to encode the temporal dynamics, and a convolutional decoder to integrate spatio-temporal features and predict the future frame. Next, a dual discriminator based adversarial training procedure, which jointly considers an image discriminator that can maintain the local consistency at frame-level and a video discriminator that can enforce the global coherence of temporal dynamics, is employed to enhance the future frame prediction. Finally, the prediction error is used to identify abnormal video frames. Thoroughly empirical studies on three public video anomaly detection datasets, i.e., UCSD Ped2, CUHK Avenue, and Shanghai Tech Campus, demonstrate the effectiveness of the proposed adversarial spatio-temporal modeling framework.