Triggering hallucinations in model-based MRI reconstruction via adversarial perturbations

TL;DR

This paper investigates how generative models for MRI reconstruction are vulnerable to adversarial noise that causes hallucinations, highlighting the need for better detection and robustness methods in medical imaging.

Contribution

It demonstrates the susceptibility of state-of-the-art MRI reconstruction models to adversarial perturbations that induce hallucinations, and emphasizes the importance of developing new detection techniques.

Findings

Models are highly sensitive to small adversarial perturbations.

Hallucinations cannot be reliably detected with traditional image quality metrics.

Adversarial training may help reduce hallucination occurrence.

Abstract

Generative models are increasingly used to improve the quality of medical imaging, such as reconstruction of magnetic resonance images and computed tomography. However, it is well-known that such models are susceptible to hallucinations: they may insert features into the reconstructed image which are not actually present in the original image. In a medical setting, such hallucinations may endanger patient health as they can lead to incorrect diagnoses. In this work, we aim to quantify the extent to which state-of-the-art generative models suffer from hallucinations in the context of magnetic resonance image reconstruction. Specifically, we craft adversarial perturbations resembling random noise for the unprocessed input images which induce hallucinations when reconstructed using a generative model. We perform this evaluation on the brain and knee images from the fastMRI data set using UNet and end-to-end VarNet architectures to reconstruct the images. Our results show that these models are highly susceptible to small perturbations and can be easily coaxed into producing hallucinations. This fragility may partially explain why hallucinations occur in the first place and suggests that a carefully constructed adversarial training routine may reduce their prevalence. Moreover, these hallucinations cannot be reliably detected using traditional image quality metrics. Novel approaches will therefore need to be developed to detect when hallucinations have occurred.