Learning to Be a Transformer to Pinpoint Anomalies

TL;DR

This paper introduces a Teacher-Student framework using high-resolution images and pre-trained features to improve anomaly detection and segmentation, especially for tiny defects, with faster processing and state-of-the-art results.

Contribution

The paper presents a novel Teacher-Student paradigm leveraging pre-trained vision Transformers and shallow MLPs to enhance high-resolution anomaly detection and segmentation.

Findings

Achieves state-of-the-art performance on MVTec AD.

Runs significantly faster than existing methods.

Excels at detecting both large and tiny anomalies.

Abstract

To efficiently deploy strong, often pre-trained feature extractors, recent Industrial Anomaly Detection and Segmentation (IADS) methods process low-resolution images, e.g., 224x224 pixels, obtained by downsampling the original input images. However, while numerous industrial applications demand the identification of both large and small defects, downsampling the input image to a low resolution may hinder a method's ability to pinpoint tiny anomalies. We propose a novel Teacher--Student paradigm to leverage strong pre-trained features while processing high-resolution input images very efficiently. The core idea concerns training two shallow MLPs (the Students) by nominal images so as to mimic the mappings between the patch embeddings induced by the self-attention layers of a frozen vision Transformer (the Teacher). Indeed, learning these mappings sets forth a challenging pretext task that small-capacity models are unlikely to accomplish on out-of-distribution data such as anomalous images. Our method can spot anomalies from high-resolution images and runs way faster than competitors, achieving state-of-the-art performance on MVTec AD and the best segmentation results on VisA. We also propose novel evaluation metrics to capture robustness to defect size, i.e., the ability to preserve good localisation from large anomalies to tiny ones. Evaluating our method also by these metrics reveals its neatly superior performance.