Temporal Divide-and-Conquer Anomaly Actions Localization in Semi-Supervised Videos with Hierarchical Transformer

TL;DR

This paper introduces a hierarchical transformer model that leverages temporal relations within videos to improve anomaly action localization in semi-supervised settings, outperforming existing segment-level and pseudo-labeling methods.

Contribution

It proposes a novel divide-and-conquer hierarchical transformer approach for temporal relation learning to enhance anomaly localization in semi-supervised videos.

Findings

Outperforms previous segment-level multiple-instance learning methods.

Achieves competitive results compared to pseudo-labeling approaches.

Effectively interprets actions and localizes anomalies in surveillance videos.

Abstract

Anomaly action detection and localization play an essential role in security and advanced surveillance systems. However, due to the tremendous amount of surveillance videos, most of the available data for the task is unlabeled or semi-labeled with the video class known, but the location of the anomaly event is unknown. In this work, we target anomaly localization in semi-supervised videos. While the mainstream direction in addressing this task is focused on segment-level multi-instance learning and the generation of pseudo labels, we aim to explore a promising yet unfulfilled direction to solve the problem by learning the temporal relations within videos in order to locate anomaly events. To this end, we propose a hierarchical transformer model designed to evaluate the significance of observed actions in anomalous videos with a divide-and-conquer strategy along the temporal axis. Our approach segments a parent video hierarchically into multiple temporal children instances and measures the influence of the children nodes in classifying the abnormality of the parent video. Evaluating our model on two well-known anomaly detection datasets, UCF-crime and ShanghaiTech, proves its ability to interpret the observed actions within videos and localize the anomalous ones. Our proposed approach outperforms previous works relying on segment-level multiple-instance learning approaches while reaching a promising performance compared to the more recent pseudo-labeling-based approaches.