View-Invariant Pixelwise Anomaly Detection in Multi-object Scenes with Adaptive View Synthesis

TL;DR

This paper introduces Scene Anomaly Detection as an unsupervised pixel-wise localization task in multi-object, multi-view environments, proposing a new dataset, augmentation strategies, and the OmniAD method to improve detection accuracy.

Contribution

It formalizes Scene AD, creates ToyCity dataset, develops view synthesis augmentations, and refines Reverse Distillation into OmniAD for better anomaly localization.

Findings

ToyCity dataset reveals baseline challenges

Synthetic augmentation provides minor improvements

OmniAD with augmentation significantly boosts performance

Abstract

The built environment, encompassing critical infrastructure such as bridges and buildings, requires diligent monitoring of unexpected anomalies or deviations from a normal state in captured imagery. Anomaly detection methods could aid in automating this task; however, deploying anomaly detection effectively in such environments presents significant challenges that have not been evaluated before. These challenges include camera viewpoints that vary, the presence of multiple objects within a scene, and the absence of labeled anomaly data for training. To address these comprehensively, we introduce and formalize Scene Anomaly Detection (Scene AD) as the task of unsupervised, pixel-wise anomaly localization under these specific real-world conditions. Evaluating progress in Scene AD required the development of ToyCity, the first multi-object, multi-view real-image dataset, for unsupervised anomaly detection. Our initial evaluations using ToyCity revealed that established anomaly detection baselines struggle to achieve robust pixel-level localization. To address this, two data augmentation strategies were created to generate additional synthetic images of non-anomalous regions to enhance generalizability. However, the addition of these synthetic images alone only provided minor improvements. Thus, OmniAD, a refinement of the Reverse Distillation methodology, was created to establish a stronger baseline. Our experiments demonstrate that OmniAD, when used with augmented views, yields a 64.33\% increase in pixel-wise \(F_1\) score over Reverse Distillation with no augmentation. Collectively, this work offers the Scene AD task definition, the ToyCity benchmark, the view synthesis augmentation approaches, and the OmniAD method. Project Page: https://drags99.github.io/OmniAD/