Masked Transformer for image Anomaly Localization

TL;DR

This paper introduces a Vision Transformer-based model with patch masking for image anomaly localization, improving detection accuracy by reconstructing patches from surrounding data rather than the patch itself.

Contribution

The paper proposes a novel transformer architecture utilizing multi-resolution patch masking for more effective anomaly detection in images.

Findings

Achieved competitive results on MVTec and head CT datasets.

Multi-resolution patches significantly improve detection performance.

Outperforms several existing state-of-the-art methods.

Abstract

Image anomaly detection consists in detecting images or image portions that are visually different from the majority of the samples in a dataset. The task is of practical importance for various real-life applications like biomedical image analysis, visual inspection in industrial production, banking, traffic management, etc. Most of the current deep learning approaches rely on image reconstruction: the input image is projected in some latent space and then reconstructed, assuming that the network (mostly trained on normal data) will not be able to reconstruct the anomalous portions. However, this assumption does not always hold. We thus propose a new model based on the Vision Transformer architecture with patch masking: the input image is split in several patches, and each patch is reconstructed only from the surrounding data, thus ignoring the potentially anomalous information contained in the patch itself. We then show that multi-resolution patches and their collective embeddings provide a large improvement in the model's performance compared to the exclusive use of the traditional square patches. The proposed model has been tested on popular anomaly detection datasets such as MVTec and head CT and achieved good results when compared to other state-of-the-art approaches.