M2S2L: Mamba-based Multi-Scale Spatial-temporal Learning for Video Anomaly Detection

TL;DR

This paper introduces M2S2L, a hierarchical multi-scale spatial-temporal learning framework for video anomaly detection that balances high accuracy with computational efficiency, suitable for real-time surveillance.

Contribution

It proposes a novel Mamba-based multi-scale spatial-temporal model with feature decomposition for improved behavioral modeling in video anomaly detection.

Findings

Achieves high frame-level AUCs on benchmark datasets.

Maintains real-time inference speed of 45 FPS.

Operates efficiently with 20.1G FLOPs.

Abstract

Video anomaly detection (VAD) is an essential task in the image processing community with prospects in video surveillance, which faces fundamental challenges in balancing detection accuracy with computational efficiency. As video content becomes increasingly complex with diverse behavioral patterns and contextual scenarios, traditional VAD approaches struggle to provide robust assessment for modern surveillance systems. Existing methods either lack comprehensive spatial-temporal modeling or require excessive computational resources for real-time applications. In this regard, we present a Mamba-based multi-scale spatial-temporal learning (M2S2L) framework in this paper. The proposed method employs hierarchical spatial encoders operating at multiple granularities and multi-temporal encoders capturing motion dynamics across different time scales. We also introduce a feature decomposition mechanism to enable task-specific optimization for appearance and motion reconstruction, facilitating more nuanced behavioral modeling and quality-aware anomaly assessment. Experiments on three benchmark datasets demonstrate that M2S2L framework achieves 98.5%, 92.1%, and 77.9% frame-level AUCs on UCSD Ped2, CUHK Avenue, and ShanghaiTech respectively, while maintaining efficiency with 20.1G FLOPs and 45 FPS inference speed, making it suitable for practical surveillance deployment.