A Self-Reasoning Framework for Anomaly Detection Using Video-Level Labels

TL;DR

This paper introduces a weakly supervised deep learning framework for video anomaly detection that uses only video-level labels and self-reasoning with clustering to improve accuracy, outperforming existing methods.

Contribution

It proposes a novel self-reasoning framework that leverages pseudo labels from clustering to enhance anomaly detection using only high-level video labels.

Findings

Outperforms state-of-the-art methods on UCF-crime, ShanghaiTech, and UCSD Ped2 datasets.

Effectively mitigates label noise through clustering-based pseudo labels.

Demonstrates robustness in detecting anomalies with only video-level supervision.

Abstract

Anomalous event detection in surveillance videos is a challenging and practical research problem among image and video processing community. Compared to the frame-level annotations of anomalous events, obtaining video-level annotations is quite fast and cheap though such high-level labels may contain significant noise. More specifically, an anomalous labeled video may actually contain anomaly only in a short duration while the rest of the video frames may be normal. In the current work, we propose a weakly supervised anomaly detection framework based on deep neural networks which is trained in a self-reasoning fashion using only video-level labels. To carry out the self-reasoning based training, we generate pseudo labels by using binary clustering of spatio-temporal video features which helps in mitigating the noise present in the labels of anomalous videos. Our proposed formulation encourages both the main network and the clustering to complement each other in achieving the goal of more accurate anomaly detection. The proposed framework has been evaluated on publicly available real-world anomaly detection datasets including UCF-crime, ShanghaiTech and UCSD Ped2. The experiments demonstrate superiority of our proposed framework over the current state-of-the-art methods.