Deep Learning for Video Anomaly Detection: A Review

TL;DR

This comprehensive review surveys deep learning methods for video anomaly detection across various supervision levels, highlighting recent advances, datasets, and future research directions in the field.

Contribution

It provides an extensive taxonomy and comparison of deep learning-based VAD methods, including recent large pre-trained models, filling gaps left by previous reviews.

Findings

Deep learning has significantly advanced VAD performance.

A well-organized taxonomy of supervision levels aids method comparison.

The review discusses datasets, codes, and metrics for VAD evaluation.

Abstract

Video anomaly detection (VAD) aims to discover behaviors or events deviating from the normality in videos. As a long-standing task in the field of computer vision, VAD has witnessed much good progress. In the era of deep learning, with the explosion of architectures of continuously growing capability and capacity, a great variety of deep learning based methods are constantly emerging for the VAD task, greatly improving the generalization ability of detection algorithms and broadening the application scenarios. Therefore, such a multitude of methods and a large body of literature make a comprehensive survey a pressing necessity. In this paper, we present an extensive and comprehensive research review, covering the spectrum of five different categories, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD, and we also delve into the latest VAD works based on pre-trained large models, remedying the limitations of past reviews in terms of only focusing on semi-supervised VAD and small model based methods. For the VAD task with different levels of supervision, we construct a well-organized taxonomy, profoundly discuss the characteristics of different types of methods, and show their performance comparisons. In addition, this review involves the public datasets, open-source codes, and evaluation metrics covering all the aforementioned VAD tasks. Finally, we provide several important research directions for the VAD community.