Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Model

TL;DR

This paper introduces a semi-supervised hard attention model using reinforcement learning for video violence detection, achieving state-of-the-art accuracy without requiring attention annotations.

Contribution

The study presents a novel semi-supervised hard attention mechanism with reinforcement learning that improves video violence detection accuracy and reduces annotation requirements.

Findings

Achieved 90.4% accuracy on RWF dataset.

Achieved 98.7% accuracy on Hockey dataset.

Outperformed existing models with a semi-supervised approach.

Abstract

The significant growth of surveillance camera networks necessitates scalable AI solutions to efficiently analyze the large amount of video data produced by these networks. As a typical analysis performed on surveillance footage, video violence detection has recently received considerable attention. The majority of research has focused on improving existing methods using supervised methods, with little, if any, attention to the semi-supervised learning approaches. In this study, a reinforcement learning model is introduced that can outperform existing models through a semi-supervised approach. The main novelty of the proposed method lies in the introduction of a semi-supervised hard attention mechanism. Using hard attention, the essential regions of videos are identified and separated from the non-informative parts of the data. A model's accuracy is improved by removing redundant data and focusing on useful visual information in a higher resolution. Implementing hard attention mechanisms using semi-supervised reinforcement learning algorithms eliminates the need for attention annotations in video violence datasets, thus making them readily applicable. The proposed model utilizes a pre-trained I3D backbone to accelerate and stabilize the training process. The proposed model achieved state-of-the-art accuracy of 90.4% and 98.7% on RWF and Hockey datasets, respectively.