JOSENet: A Joint Stream Embedding Network for Violence Detection in Surveillance Videos

TL;DR

JOSENet is a self-supervised deep learning framework that efficiently detects violence in surveillance videos by processing dual video streams, achieving high accuracy with lower computational costs and fewer frames.

Contribution

The paper introduces JOSENet, a novel self-supervised model that improves violence detection in surveillance videos while reducing memory and computational requirements.

Findings

Outperforms state-of-the-art methods in violence detection

Uses only one-fourth of frames per video segment

Operates at a lower frame rate with high accuracy

Abstract

The increasing proliferation of video surveillance cameras and the escalating demand for crime prevention have intensified interest in the task of violence detection within the research community. Compared to other action recognition tasks, violence detection in surveillance videos presents additional issues, such as the wide variety of real fight scenes. Unfortunately, existing datasets for violence detection are relatively small in comparison to those for other action recognition tasks. Moreover, surveillance footage often features different individuals in each video and varying backgrounds for each camera. In addition, fast detection of violent actions in real-life surveillance videos is crucial to prevent adverse outcomes, thus necessitating models that are optimized for reduced memory usage and computational costs. These challenges complicate the application of traditional action recognition methods. To tackle all these issues, we introduce JOSENet, a novel self-supervised framework that provides outstanding performance for violence detection in surveillance videos. The proposed model processes two spatiotemporal video streams, namely RGB frames and optical flows, and incorporates a new regularized self-supervised learning approach for videos. JOSENet demonstrates improved performance compared to state-of-the-art methods, while utilizing only one-fourth of the frames per video segment and operating at a reduced frame rate. The source code is available at https://github.com/ispamm/JOSENet.