Learning to Tell Apart: Weakly Supervised Video Anomaly Detection via Disentangled Semantic Alignment

TL;DR

This paper introduces DSANet, a novel weakly-supervised video anomaly detection model that explicitly separates normal and abnormal features at multiple levels, improving fine-grained classification and reducing category confusion.

Contribution

The paper proposes a Disentangled Semantic Alignment Network with coarse- and fine-grained modules, including normality modeling and contrastive semantic alignment, to enhance anomaly detection accuracy.

Findings

Outperforms state-of-the-art on XD-Violence and UCF-Crime benchmarks.

Effectively separates normal and abnormal patterns at multiple levels.

Improves fine-grained classification and reduces category confusion.

Abstract

Recent advancements in weakly-supervised video anomaly detection have achieved remarkable performance by applying the multiple instance learning paradigm based on multimodal foundation models such as CLIP to highlight anomalous instances and classify categories. However, their objectives may tend to detect the most salient response segments, while neglecting to mine diverse normal patterns separated from anomalies, and are prone to category confusion due to similar appearance, leading to unsatisfactory fine-grained classification results. Therefore, we propose a novel Disentangled Semantic Alignment Network (DSANet) to explicitly separate abnormal and normal features from coarse-grained and fine-grained aspects, enhancing the distinguishability. Specifically, at the coarse-grained level, we introduce a self-guided normality modeling branch that reconstructs input video features under the guidance of learned normal prototypes, encouraging the model to exploit normality cues inherent in the video, thereby improving the temporal separation of normal patterns and anomalous events. At the fine-grained level, we present a decoupled contrastive semantic alignment mechanism, which first temporally decomposes each video into event-centric and background-centric components using frame-level anomaly scores and then applies visual-language contrastive learning to enhance class-discriminative representations. Comprehensive experiments on two standard benchmarks, namely XD-Violence and UCF-Crime, demonstrate that DSANet outperforms existing state-of-the-art methods.