RefineVAD: Semantic-Guided Feature Recalibration for Weakly Supervised Video Anomaly Detection

TL;DR

RefineVAD introduces a dual-module framework that combines motion salience and semantic category priors to improve weakly-supervised video anomaly detection by modeling both temporal dynamics and semantic structures.

Contribution

It proposes a novel semantic-guided feature recalibration framework that jointly models motion and semantic information, addressing limitations of previous methods that treat anomalies as a single category.

Findings

Outperforms existing methods on WVAD benchmark

Effectively models diverse anomaly types through semantic integration

Demonstrates the importance of semantic context in anomaly detection

Abstract

Weakly-Supervised Video Anomaly Detection aims to identify anomalous events using only video-level labels, balancing annotation efficiency with practical applicability. However, existing methods often oversimplify the anomaly space by treating all abnormal events as a single category, overlooking the diverse semantic and temporal characteristics intrinsic to real-world anomalies. Inspired by how humans perceive anomalies, by jointly interpreting temporal motion patterns and semantic structures underlying different anomaly types, we propose RefineVAD, a novel framework that mimics this dual-process reasoning. Our framework integrates two core modules. The first, Motion-aware Temporal Attention and Recalibration (MoTAR), estimates motion salience and dynamically adjusts temporal focus via shift-based attention and global Transformer-based modeling. The second, Category-Oriented Refinement (CORE), injects soft anomaly category priors into the representation space by aligning segment-level features with learnable category prototypes through cross-attention. By jointly leveraging temporal dynamics and semantic structure, explicitly models both "how" motion evolves and "what" semantic category it resembles. Extensive experiments on WVAD benchmark validate the effectiveness of RefineVAD and highlight the importance of integrating semantic context to guide feature refinement toward anomaly-relevant patterns.