Flashback: Memory-Driven Zero-shot, Real-time Video Anomaly Detection

TL;DR

Flashback introduces a zero-shot, real-time video anomaly detection method inspired by human memory, using large language models offline to create a scene memory, enabling instant online anomaly detection without heavy computation.

Contribution

The paper presents Flashback, a novel zero-shot, real-time VAD framework that leverages large language models offline to enable fast online anomaly detection without relying on real anomaly data.

Findings

Achieves 87.3 AUC on UCF-Crime dataset.

Attains 75.1 AP on XD-Violence dataset.

Outperforms prior zero-shot VAD methods significantly.

Abstract

Video Anomaly Detection (VAD) automatically identifies anomalous events from video, mitigating the need for human operators in large-scale surveillance deployments. However, two fundamental obstacles hinder real-world adoption: domain dependency and real-time constraints -- requiring near-instantaneous processing of incoming video. To this end, we propose Flashback, a zero-shot and real-time video anomaly detection paradigm. Inspired by the human cognitive mechanism of instantly judging anomalies and reasoning in current scenes based on past experience, Flashback operates in two stages: Recall and Respond. In the offline recall stage, an off-the-shelf LLM builds a pseudo-scene memory of both normal and anomalous captions without any reliance on real anomaly data. In the online respond stage, incoming video segments are embedded and matched against this memory via similarity search. By eliminating all LLM calls at inference time, Flashback delivers real-time VAD even on a consumer-grade GPU. On two large datasets from real-world surveillance scenarios, UCF-Crime and XD-Violence, we achieve 87.3 AUC (+7.0 pp) and 75.1 AP (+13.1 pp), respectively, outperforming prior zero-shot VAD methods by large margins.