Adversarially-Aware Architecture Design for Robust Medical AI Systems

TL;DR

This paper investigates the vulnerabilities of medical AI systems to adversarial attacks, demonstrating their impact on accuracy and evaluating defenses like adversarial training, emphasizing the need for comprehensive resilience strategies.

Contribution

It provides empirical analysis of adversarial threats in medical AI, benchmarks defense methods, and discusses the balance between robustness and performance in healthcare applications.

Findings

Adversarial attacks significantly reduce model accuracy.

Defenses like adversarial training partially mitigate attacks.

Trade-offs exist between robustness and model performance.

Abstract

Adversarial attacks pose a severe risk to AI systems used in healthcare, capable of misleading models into dangerous misclassifications that can delay treatments or cause misdiagnoses. These attacks, often imperceptible to human perception, threaten patient safety, particularly in underserved populations. Our study explores these vulnerabilities through empirical experimentation on a dermatological dataset, where adversarial methods significantly reduce classification accuracy. Through detailed threat modeling, experimental benchmarking, and model evaluation, we demonstrate both the severity of the threat and the partial success of defenses like adversarial training and distillation. Our results show that while defenses reduce attack success rates, they must be balanced against model performance on clean data. We conclude with a call for integrated technical, ethical, and policy-based approaches to build more resilient, equitable AI in healthcare.