Scaling Laws for Black box Adversarial Attacks

TL;DR

This paper uncovers a universal log-linear scaling law for black-box adversarial attack success rates, demonstrating that increasing ensemble size significantly enhances attack effectiveness across various models and defenses.

Contribution

It introduces the first large-scale empirical study revealing a fundamental scaling law for ensemble-based black-box attacks, supported by theoretical analysis and extensive experiments.

Findings

Attack success rate scales linearly with the logarithm of ensemble size

Scaling improves transferability across classifiers, defenses, and MLLMs

Achieves over 80% success on proprietary models like GPT-4o

Abstract

Adversarial examples exhibit cross-model transferability, enabling threatening black-box attacks on commercial models. Model ensembling, which attacks multiple surrogate models, is a known strategy to improve this transferability. However, prior studies typically use small, fixed ensembles, which leaves open an intriguing question of whether scaling the number of surrogate models can further improve black-box attacks. In this work, we conduct the first large-scale empirical study of this question. We show that by resolving gradient conflict with advanced optimizers, we discover a robust and universal log-linear scaling law through both theoretical analysis and empirical evaluations: the Attack Success Rate (ASR) scales linearly with the logarithm of the ensemble size $T$. We rigorously verify this law across standard classifiers, SOTA defenses, and MLLMs, and find that scaling distills robust, semantic features of the target class. Consequently, we apply this fundamental insight to benchmark SOTA MLLMs. This reveals both the attack's devastating power and a clear robustness hierarchy: we achieve 80\%+ transfer attack success rate on proprietary models like GPT-4o, while also highlighting the exceptional resilience of Claude-3.5-Sonnet. Our findings urge a shift in focus for robustness evaluation: from designing intricate algorithms on small ensembles to understanding the principled and powerful threat of scaling.