Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detection

TL;DR

This paper introduces UR-DMU, a novel model for weakly supervised video anomaly detection that learns normal and abnormal representations using dual memory banks and uncertainty regulation, improving detection accuracy.

Contribution

The paper proposes an Uncertainty Regulated Dual Memory Units model with a GL-MHSA module and separate memory banks for normal and abnormal data, enhancing discriminative feature learning.

Findings

Outperforms state-of-the-art on XD-Violence and UCF-Crime datasets.

Effectively separates normal and abnormal prototypes.

Robust to noise from scene changes and camera switching.

Abstract

Learning discriminative features for effectively separating abnormal events from normality is crucial for weakly supervised video anomaly detection (WS-VAD) tasks. Existing approaches, both video and segment-level label oriented, mainly focus on extracting representations for anomaly data while neglecting the implication of normal data. We observe that such a scheme is sub-optimal, i.e., for better distinguishing anomaly one needs to understand what is a normal state, and may yield a higher false alarm rate. To address this issue, we propose an Uncertainty Regulated Dual Memory Units (UR-DMU) model to learn both the representations of normal data and discriminative features of abnormal data. To be specific, inspired by the traditional global and local structure on graph convolutional networks, we introduce a Global and Local Multi-Head Self Attention (GL-MHSA) module for the Transformer network to obtain more expressive embeddings for capturing associations in videos. Then, we use two memory banks, one additional abnormal memory for tackling hard samples, to store and separate abnormal and normal prototypes and maximize the margins between the two representations. Finally, we propose an uncertainty learning scheme to learn the normal data latent space, that is robust to noise from camera switching, object changing, scene transforming, etc. Extensive experiments on XD-Violence and UCF-Crime datasets demonstrate that our method outperforms the state-of-the-art methods by a sizable margin.