MixedTeacher : Knowledge Distillation for fast inference textural anomaly detection

TL;DR

This paper introduces MixedTeacher, a knowledge distillation approach for texture anomaly detection that improves detection accuracy and inference speed by using a novel student-teacher architecture with optimized layer selection and dual teachers.

Contribution

It proposes a reduced student network with optimal layer selection and a dual-teacher architecture to enhance anomaly detection and localization accuracy.

Findings

Fast inference compared to state-of-the-art methods

High defect detection capability in various textures

Effective network bias reduction with dual teachers

Abstract

For a very long time, unsupervised learning for anomaly detection has been at the heart of image processing research and a stepping stone for high performance industrial automation process. With the emergence of CNN, several methods have been proposed such as Autoencoders, GAN, deep feature extraction, etc. In this paper, we propose a new method based on the promising concept of knowledge distillation which consists of training a network (the student) on normal samples while considering the output of a larger pretrained network (the teacher). The main contributions of this paper are twofold: First, a reduced student architecture with optimal layer selection is proposed, then a new Student-Teacher architecture with network bias reduction combining two teachers is proposed in order to jointly enhance the performance of anomaly detection and its localization accuracy. The proposed texture anomaly detector has an outstanding capability to detect defects in any texture and a fast inference time compared to the SOTA methods.