Useful nonrobust features are ubiquitous in biomedical images

TL;DR

This paper investigates the role of nonrobust features in medical imaging deep learning models, showing they improve in-distribution accuracy but harm out-of-distribution robustness, highlighting a trade-off.

Contribution

It demonstrates the prevalence of nonrobust features in biomedical images and analyzes their impact on model accuracy and robustness under distribution shifts.

Findings

Models trained on nonrobust features perform well in-distribution.

Adversarial training enhances robustness but reduces accuracy.

Nonrobust features cause a robustness-accuracy trade-off.

Abstract

We study whether deep networks for medical imaging learn useful nonrobust features - predictive input patterns that are not human interpretable and highly susceptible to small adversarial perturbations - and how these features impact test performance. We show that models trained only on nonrobust features achieve well above chance accuracy across five MedMNIST classification tasks, confirming their predictive value in-distribution. Conversely, adversarially trained models that primarily rely on robust features sacrifice in-distribution accuracy but yield markedly better performance under controlled distribution shifts (MedMNIST-C). Overall, nonrobust features boost standard accuracy yet degrade out-of-distribution performance, revealing a practical robustness-accuracy trade-off in medical imaging classification tasks that should be tailored to the requirements of the deployment setting.