RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defenses

TL;DR

This paper evaluates the effectiveness of recent black-box adversarial attacks against state-of-the-art robust models on ImageNet, revealing their limited success and highlighting the importance of robustness alignment.

Contribution

It introduces a framework for assessing black-box attack effectiveness on robust models and provides empirical insights into their limitations and factors influencing attack success.

Findings

Black-box attacks struggle against simple adversarially trained models

Robust models resistant to white-box attacks also resist black-box attacks

Robustness alignment between surrogate and target models affects transfer attack success

Abstract

Although adversarial robustness has been extensively studied in white-box settings, recent advances in black-box attacks (including transfer- and query-based approaches) are primarily benchmarked against weak defenses, leaving a significant gap in the evaluation of their effectiveness against more recent and moderate robust models (e.g., those featured in the Robustbench leaderboard). In this paper, we question this lack of attention from black-box attacks to robust models. We establish a framework to evaluate the effectiveness of recent black-box attacks against both top-performing and standard defense mechanisms, on the ImageNet dataset. Our empirical evaluation reveals the following key findings: (1) the most advanced black-box attacks struggle to succeed even against simple adversarially trained models; (2) robust models that are optimized to withstand strong white-box attacks, such as AutoAttack, also exhibits enhanced resilience against black-box attacks; and (3) robustness alignment between the surrogate models and the target model plays a key factor in the success rate of transfer-based attacks