RobustA: Robust Anomaly Detection in Multimodal Data

TL;DR

This paper introduces RobustA, a new dataset and method for multimodal anomaly detection that is resilient to corrupted audio and visual data, advancing real-world applications.

Contribution

It presents the first comprehensive study on the effects of modality corruption and proposes a robust detection method with a shared representation and dynamic weighting.

Findings

RobustA dataset effectively captures corrupted modality scenarios.

The proposed method demonstrates high resilience to corrupted data.

The approach improves anomaly detection accuracy under adverse conditions.

Abstract

In recent years, multimodal anomaly detection methods have demonstrated remarkable performance improvements over video-only models. However, real-world multimodal data is often corrupted due to unforeseen environmental distortions. In this paper, we present the first-of-its-kind work that comprehensively investigates the adverse effects of corrupted modalities on multimodal anomaly detection task. To streamline this work, we propose RobustA, a carefully curated evaluation dataset to systematically observe the impacts of audio and visual corruptions on the overall effectiveness of anomaly detection systems. Furthermore, we propose a multimodal anomaly detection method, which shows notable resilience against corrupted modalities. The proposed method learns a shared representation space for different modalities and employs a dynamic weighting scheme during inference based on the estimated level of corruption. Our work represents a significant step forward in enabling the real-world application of multimodal anomaly detection, addressing situations where the likely events of modality corruptions occur. The proposed evaluation dataset with corrupted modalities and respective extracted features will be made publicly available.