Multimodal Task Representation Memory Bank vs. Catastrophic Forgetting in Anomaly Detection

TL;DR

This paper introduces MTRMB, a novel memory bank approach for unsupervised continuous anomaly detection that effectively mitigates catastrophic forgetting and enhances multi-modal feature representation using innovative cross-modal interaction and contrastive learning techniques.

Contribution

The paper proposes the Multimodal Task Representation Memory Bank (MTRMB) with key innovations like KPMK and RSCL to improve multi-modal anomaly detection and reduce forgetting in unsupervised learning.

Findings

MTRMB achieves 0.921 detection accuracy on MVtec AD and VisA datasets.

Significantly outperforms existing state-of-the-art methods.

Reduces catastrophic forgetting in multi-task unsupervised anomaly detection.

Abstract

Unsupervised Continuous Anomaly Detection (UCAD) faces significant challenges in multi-task representation learning, with existing methods suffering from incomplete representation and catastrophic forgetting. Unlike supervised models, unsupervised scenarios lack prior information, making it difficult to effectively distinguish redundant and complementary multimodal features. To address this, we propose the Multimodal Task Representation Memory Bank (MTRMB) method through two key technical innovations: A Key-Prompt-Multimodal Knowledge (KPMK) mechanism that uses concise key prompts to guide cross-modal feature interaction between BERT and ViT. Refined Structure-based Contrastive Learning (RSCL) leveraging Grounding DINO and SAM to generate precise segmentation masks, pulling features of the same structural region closer while pushing different structural regions apart. Experiments on MVtec AD and VisA datasets demonstrate MTRMB's superiority, achieving an average detection accuracy of 0.921 at the lowest forgetting rate, significantly outperforming state-of-the-art methods. We plan to open source on GitHub.