Unsupervised Anomaly Detection in Medical Images with a Memory-augmented Multi-level Cross-attentional Masked Autoencoder

TL;DR

This paper introduces MemMC-MAE, a transformer-based unsupervised anomaly detection method that uses memory and multi-level cross-attention to improve detection and localization of anomalies in medical images, outperforming existing methods.

Contribution

The paper proposes a novel memory-augmented multi-level cross-attentional masked autoencoder for unsupervised anomaly detection in medical images, addressing low-reconstruction error issues of previous reconstruction-based methods.

Findings

Achieves state-of-the-art results on colonoscopy, pneumonia, and COVID-19 chest X-ray datasets.

Effective in both anomaly detection and localization tasks.

Outperforms existing reconstruction-based anomaly detection methods.

Abstract

Unsupervised anomaly detection (UAD) aims to find anomalous images by optimising a detector using a training set that contains only normal images. UAD approaches can be based on reconstruction methods, self-supervised approaches, and Imagenet pre-trained models. Reconstruction methods, which detect anomalies from image reconstruction errors, are advantageous because they do not rely on the design of problem-specific pretext tasks needed by self-supervised approaches, and on the unreliable translation of models pre-trained from non-medical datasets. However, reconstruction methods may fail because they can have low reconstruction errors even for anomalous images. In this paper, we introduce a new reconstruction-based UAD approach that addresses this low-reconstruction error issue for anomalous images. Our UAD approach, the memory-augmented multi-level cross-attentional masked autoencoder (MemMC-MAE), is a transformer-based approach, consisting of a novel memory-augmented self-attention operator for the encoder and a new multi-level cross-attention operator for the decoder. MemMCMAE masks large parts of the input image during its reconstruction, reducing the risk that it will produce low reconstruction errors because anomalies are likely to be masked and cannot be reconstructed. However, when the anomaly is not masked, then the normal patterns stored in the encoder's memory combined with the decoder's multi-level cross attention will constrain the accurate reconstruction of the anomaly. We show that our method achieves SOTA anomaly detection and localisation on colonoscopy, pneumonia, and covid-19 chest x-ray datasets.