X-MAN: Explaining multiple sources of anomalies in video

TL;DR

This paper introduces X-MAN, a novel approach for detecting anomalies in videos that emphasizes interpretability by explaining anomalies through object interactions and high-level concepts, while maintaining competitive performance.

Contribution

It presents an interpretable probabilistic anomaly detector, incorporates object interactions for the first time, and introduces a new task and dataset for explaining video anomalies.

Findings

Competitive performance on public datasets

Effective explanation of anomalies via object interactions

First to consider object interactions explicitly in anomaly detection

Abstract

Our objective is to detect anomalies in video while also automatically explaining the reason behind the detector's response. In a practical sense, explainability is crucial for this task as the required response to an anomaly depends on its nature and severity. However, most leading methods (based on deep neural networks) are not interpretable and hide the decision making process in uninterpretable feature representations. In an effort to tackle this problem we make the following contributions: (1) we show how to build interpretable feature representations suitable for detecting anomalies with state of the art performance, (2) we propose an interpretable probabilistic anomaly detector which can describe the reason behind it's response using high level concepts, (3) we are the first to directly consider object interactions for anomaly detection and (4) we propose a new task of explaining anomalies and release a large dataset for evaluating methods on this task. Our method competes well with the state of the art on public datasets while also providing anomaly explanation based on objects and their interactions.