Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge

TL;DR

This paper proposes a method that combines generic CNN-based knowledge with environment-specific detectors to improve the detection and recounting of abnormal events in videos, aiding human understanding and verification.

Contribution

It introduces an integrated approach that leverages CNNs for semantic understanding and environment-dependent detectors for effective abnormal event detection and recounting.

Findings

Outperforms state-of-the-art on Avenue and UCSD Ped2 benchmarks.

Effective in detecting and recounting abnormal events.

Provides promising results in event recounting.

Abstract

This paper addresses the problem of joint detection and recounting of abnormal events in videos. Recounting of abnormal events, i.e., explaining why they are judged to be abnormal, is an unexplored but critical task in video surveillance, because it helps human observers quickly judge if they are false alarms or not. To describe the events in the human-understandable form for event recounting, learning generic knowledge about visual concepts (e.g., object and action) is crucial. Although convolutional neural networks (CNNs) have achieved promising results in learning such concepts, it remains an open question as to how to effectively use CNNs for abnormal event detection, mainly due to the environment-dependent nature of the anomaly detection. In this paper, we tackle this problem by integrating a generic CNN model and environment-dependent anomaly detectors. Our approach first learns CNN with multiple visual tasks to exploit semantic information that is useful for detecting and recounting abnormal events. By appropriately plugging the model into anomaly detectors, we can detect and recount abnormal events while taking advantage of the discriminative power of CNNs. Our approach outperforms the state-of-the-art on Avenue and UCSD Ped2 benchmarks for abnormal event detection and also produces promising results of abnormal event recounting.