Explainability Matters: Backdoor Attacks on Medical Imaging

TL;DR

This paper demonstrates that backdoor attacks can be effectively executed on medical imaging models with minimal pixel perturbations, highlighting the importance of explainability for detecting such vulnerabilities in clinical AI systems.

Contribution

It reveals the vulnerability of medical imaging neural networks to backdoor attacks using small pixel triggers and emphasizes the role of explainability in identifying these backdoors.

Findings

A 3x3 pixel trigger can achieve up to 1.00 AUROC on infected images.

Backdoored models maintain high performance on clean images (up to 0.85 AUROC).

Explainability helps detect spatially localized backdoors during inference.

Abstract

Deep neural networks have been shown to be vulnerable to backdoor attacks, which could be easily introduced to the training set prior to model training. Recent work has focused on investigating backdoor attacks on natural images or toy datasets. Consequently, the exact impact of backdoors is not yet fully understood in complex real-world applications, such as in medical imaging where misdiagnosis can be very costly. In this paper, we explore the impact of backdoor attacks on a multi-label disease classification task using chest radiography, with the assumption that the attacker can manipulate the training dataset to execute the attack. Extensive evaluation of a state-of-the-art architecture demonstrates that by introducing images with few-pixel perturbations into the training set, an attacker can execute the backdoor successfully without having to be involved with the training procedure. A simple 3$\times$3 pixel trigger can achieve up to 1.00 Area Under the Receiver Operating Characteristic (AUROC) curve on the set of infected images. In the set of clean images, the backdoored neural network could still achieve up to 0.85 AUROC, highlighting the stealthiness of the attack. As the use of deep learning based diagnostic systems proliferates in clinical practice, we also show how explainability is indispensable in this context, as it can identify spatially localized backdoors in inference time.