Cyclostationary Statistical Models and Algorithms for Anomaly Detection Using Multi-Modal Data

TL;DR

This paper introduces a novel framework combining deep learning and cyclostationary models to detect anomalies in multi-modal data, demonstrated on CCTV and social media data for event detection.

Contribution

It proposes a new cyclostationary modeling approach for anomaly detection in multi-modal data, integrating deep neural networks and sequential algorithms.

Findings

Algorithms are asymptotically efficient.

Effective detection of a 5K run event in NYC.

Applicable to CCTV and social media data.

Abstract

A framework is proposed to detect anomalies in multi-modal data. A deep neural network-based object detector is employed to extract counts of objects and sub-events from the data. A cyclostationary model is proposed to model regular patterns of behavior in the count sequences. The anomaly detection problem is formulated as a problem of detecting deviations from learned cyclostationary behavior. Sequential algorithms are proposed to detect anomalies using the proposed model. The proposed algorithms are shown to be asymptotically efficient in a well-defined sense. The developed algorithms are applied to a multi-modal data consisting of CCTV imagery and social media posts to detect a 5K run in New York City.