Memory-augmented Online Video Anomaly Detection

TL;DR

This paper introduces MOVAD, a modular, end-to-end system for real-time video anomaly detection in autonomous vehicle scenarios, combining short-term and long-term memory modules to improve detection accuracy.

Contribution

The paper presents a novel online video anomaly detection architecture that integrates short-term and long-term memory modules, achieving state-of-the-art performance with a simple, end-to-end trainable design.

Findings

Achieved 82.17% AUC on DoTA dataset, surpassing previous methods.

Demonstrated the effectiveness of combining VST and LSTM modules for anomaly detection.

Provided a modular architecture that is easy to implement and adapt.

Abstract

The ability to understand the surrounding scene is of paramount importance for Autonomous Vehicles (AVs). This paper presents a system capable to work in an online fashion, giving an immediate response to the arise of anomalies surrounding the AV, exploiting only the videos captured by a dash-mounted camera. Our architecture, called MOVAD, relies on two main modules: a Short-Term Memory Module to extract information related to the ongoing action, implemented by a Video Swin Transformer (VST), and a Long-Term Memory Module injected inside the classifier that considers also remote past information and action context thanks to the use of a Long-Short Term Memory (LSTM) network. The strengths of MOVAD are not only linked to its excellent performance, but also to its straightforward and modular architecture, trained in a end-to-end fashion with only RGB frames with as less assumptions as possible, which makes it easy to implement and play with. We evaluated the performance of our method on Detection of Traffic Anomaly (DoTA) dataset, a challenging collection of dash-mounted camera videos of accidents. After an extensive ablation study, MOVAD is able to reach an AUC score of 82.17\%, surpassing the current state-of-the-art by +2.87 AUC. Our code will be available on https://github.com/IMPLabUniPr/movad/tree/movad_vad