Regularity Learning via Explicit Distribution Modeling for Skeletal Video Anomaly Detection

TL;DR

This paper introduces a novel framework for skeletal video anomaly detection that models pose motion explicitly and employs a spatial-temporal transformer for self-supervised learning, achieving state-of-the-art results.

Contribution

It proposes a new Motion Embedder and Spatial-Temporal Transformer for pose regularity learning in a unified framework, enhancing anomaly detection performance.

Findings

Achieves an average 4.7% AUC improvement on challenging datasets.

Validates the effectiveness of the proposed modules through extensive experiments.

Provides a pose motion representation from a probability perspective.

Abstract

Anomaly detection in surveillance videos is challenging and important for ensuring public security. Different from pixel-based anomaly detection methods, pose-based methods utilize highly-structured skeleton data, which decreases the computational burden and also avoids the negative impact of background noise. However, unlike pixel-based methods, which could directly exploit explicit motion features such as optical flow, pose-based methods suffer from the lack of alternative dynamic representation. In this paper, a novel Motion Embedder (ME) is proposed to provide a pose motion representation from the probability perspective. Furthermore, a novel task-specific Spatial-Temporal Transformer (STT) is deployed for self-supervised pose sequence reconstruction. These two modules are then integrated into a unified framework for pose regularity learning, which is referred to as Motion Prior Regularity Learner (MoPRL). MoPRL achieves the state-of-the-art performance by an average improvement of 4.7% AUC on several challenging datasets. Extensive experiments validate the versatility of each proposed module.