MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors

TL;DR

This paper introduces a multi-task learning approach using Mixture-of-Experts to improve medical image anomaly detection, outperforming existing methods and providing interpretable results.

Contribution

It proposes a novel multi-task learning framework from scratch with MoE for robust anomaly detection in medical images, surpassing state-of-the-art performance.

Findings

Outperforms all state-of-the-art methods on BMAD benchmark.

Produces interpretable anomaly maps for better diagnosis.

Effectively learns normal anatomical structures through multi-task learning.

Abstract

Anomaly detection in medical images is a challenging task, since anomalies are not typically available during training. Recent methods leverage a single pretext task coupled with a large-scale pre-trained model to reach state-of-the-art performance. Instead, we propose to learn multiple self-supervised and pseudo-labeling tasks from scratch, using a joint model based on Mixture-of-Experts (MoE). By carefully integrating multiple proxy tasks, the joint model effectively learns a robust representation of normal anatomical structures, so that anomaly scores can be derived based on how well the multi-task learner (MTL) solves each task during inference. We perform comprehensive experiments on BMAD, a recent benchmark that comprises a broad range of medical image modalities. The empirical results indicate that our multi-task learner is an effective anomaly detector, outperforming all state-of-the-art competitors on BMAD. Moreover, our model produces interpretable anomaly maps, potentially helping physicians in providing more accurate diagnoses.