AnyAD: Unified Any-Modality Anomaly Detection in Incomplete Multi-Sequence MRI

TL;DR

This paper introduces a unified framework for anomaly detection in multi-sequence MRI that is robust to missing modalities, leveraging feature alignment and semantic reconstruction to improve generalization across various modality combinations.

Contribution

The proposed AnyAD framework is the first to perform robust anomaly detection across arbitrary MRI modality combinations without re-training, using feature alignment and semantic consistency mechanisms.

Findings

Outperforms state-of-the-art AD methods on multiple datasets.

Achieves high accuracy across 7 different modality combinations.

Demonstrates strong generalization with incomplete modality data.

Abstract

Reliable anomaly detection in brain MRI remains challenging due to the scarcity of annotated abnormal cases and the frequent absence of key imaging modalities in real clinical workflows. Existing single-class or multi-class anomaly detection (AD) models typically rely on fixed modality configurations, require repetitive training, or fail to generalize to unseen modality combinations, limiting their clinical scalability. In this work, we present a unified Any-Modality AD framework that performs robust anomaly detection and localization under arbitrary MRI modality availability. The framework integrates a dual-pathway DINOv2 encoder with a feature distribution alignment mechanism that statistically aligns incomplete-modality features with full-modality representations, enabling stable inference even with severe modality dropout. To further enhance semantic consistency, we introduce an Intrinsic Normal Prototypes (INPs) extractor and an INP-guided decoder that reconstruct only normal anatomical patterns while naturally amplifying abnormal deviations. Through randomized modality masking and indirect feature completion during training, the model learns to adapt to all modality configurations without re-training. Extensive experiments on BraTS2018, MU-Glioma-Post, and Pretreat-MetsToBrain-Masks demonstrate that our approach consistently surpasses state-of-the-art industrial and medical AD baselines across 7 modality combinations, achieving superior generalization. This study establishes a scalable paradigm for multimodal medical AD under real-world, imperfect modality conditions. Our source code is available at https://github.com/wuchangw/AnyAD.