Video Anomaly Detection with Motion and Appearance Guided Patch Diffusion Model

TL;DR

This paper introduces a patch-based diffusion model for video anomaly detection that captures fine-grained local details and jointly considers appearance and motion deviations, leading to improved detection accuracy.

Contribution

It proposes a novel patch diffusion model with integrated motion and appearance conditions to better detect small and complex anomalies in videos.

Findings

Outperforms existing methods on four datasets

Effectively captures local anomalies in appearance and motion

Demonstrates robustness in challenging scenarios

Abstract

A recent endeavor in one class of video anomaly detection is to leverage diffusion models and posit the task as a generation problem, where the diffusion model is trained to recover normal patterns exclusively, thus reporting abnormal patterns as outliers. Yet, existing attempts neglect the various formations of anomaly and predict normal samples at the feature level regardless that abnormal objects in surveillance videos are often relatively small. To address this, a novel patch-based diffusion model is proposed, specifically engineered to capture fine-grained local information. We further observe that anomalies in videos manifest themselves as deviations in both appearance and motion. Therefore, we argue that a comprehensive solution must consider both of these aspects simultaneously to achieve accurate frame prediction. To address this, we introduce innovative motion and appearance conditions that are seamlessly integrated into our patch diffusion model. These conditions are designed to guide the model in generating coherent and contextually appropriate predictions for both semantic content and motion relations. Experimental results in four challenging video anomaly detection datasets empirically substantiate the efficacy of our proposed approach, demonstrating that it consistently outperforms most existing methods in detecting abnormal behaviors.