Behavior Subtraction

TL;DR

This paper introduces behavior subtraction, a novel approach that models scene dynamics as events to detect anomalies in video, overcoming limitations of traditional background subtraction methods.

Contribution

It proposes a new framework that characterizes scene dynamics as events and uses probabilistic surrogates for behavior subtraction, enhancing anomaly detection in various scenes.

Findings

Effective in anomaly detection and localization

Resilient to camera jitter and spurious motion

Content-blind, working across different object types

Abstract

Background subtraction has been a driving engine for many computer vision and video analytics tasks. Although its many variants exist, they all share the underlying assumption that photometric scene properties are either static or exhibit temporal stationarity. While this works in some applications, the model fails when one is interested in discovering {\it changes in scene dynamics} rather than those in a static background; detection of unusual pedestrian and motor traffic patterns is but one example. We propose a new model and computational framework that address this failure by considering stationary scene dynamics as a ``background'' with which observed scene dynamics are compared. Central to our approach is the concept of an {\it event}, that we define as short-term scene dynamics captured over a time window at a specific spatial location in the camera field of view. We compute events by time-aggregating motion labels, obtained by background subtraction, as well as object descriptors (e.g., object size). Subsequently, we characterize events probabilistically, but use a low-memory, low-complexity surrogates in practical implementation. Using these surrogates amounts to {\it behavior subtraction}, a new algorithm with some surprising properties. As demonstrated here, behavior subtraction is an effective tool in anomaly detection and localization. It is resilient to spurious background motion, such as one due to camera jitter, and is content-blind, i.e., it works equally well on humans, cars, animals, and other objects in both uncluttered and highly-cluttered scenes. Clearly, treating video as a collection of events rather than colored pixels opens new possibilities for video analytics.