Tuned Reverse Distillation: Enhancing Multimodal Industrial Anomaly Detection with Crossmodal Tuners

TL;DR

This paper introduces Tuned Reverse Distillation, a novel method for multimodal industrial anomaly detection that leverages multi-branch design and crossmodal tuners to improve anomaly detection and localization accuracy.

Contribution

It presents a new multi-branch distillation framework with crossmodal tuners for enhanced multimodal anomaly detection, addressing limitations of fused feature methods.

Findings

Achieves state-of-the-art performance on multimodal AD datasets.

Effectively detects anomalies in individual modalities.

Improves anomaly localization accuracy.

Abstract

Knowledge distillation (KD) has been widely studied in unsupervised image Anomaly Detection (AD), but its application to unsupervised multimodal AD remains underexplored. Existing KD-based methods for multimodal AD that use fused multimodal features to obtain teacher representations face challenges. Anomalies that only exist in one modality may not be effectively captured in the fused teacher features, leading to detection failures. Besides, these methods do not fully leverage the rich intra- and inter-modality information that are critical for effective anomaly detection. In this paper, we propose Tuned Reverse Distillation (TRD) based on Multi-branch design to realize Multimodal Industrial AD. By assigning independent branches to each modality, our method enables finer detection of anomalies within each modality. Furthermore, we enhance the interaction between modalities during the distillation process by designing two Crossmodal Tuners including Crossmodal Filter and Amplifier. With the idea of crossmodal mapping, the student network is allowed to better learn normal features while anomalies in all modalities are ensured to be effectively detected. Experimental verifications on multimodal AD datasets demonstrate that our method achieves state-of-the-art performance in multimodal anomaly detection and localization. Code is available at https://github.com/hito2448/TRD.