CARBEN: Composite Adversarial Robustness Benchmark

TL;DR

This paper introduces CARBEN, a benchmark for evaluating adversarial robustness against composite attacks that combine multiple threat models, highlighting the importance of attack order and providing tools for rapid model assessment.

Contribution

It presents a new benchmark and evaluation framework for composite adversarial attacks, expanding robustness testing beyond single threat models.

Findings

Attack order influences attack effectiveness.

Real-time inference aids in rapid robustness evaluation.

A leaderboard benchmarks models against composite attacks.

Abstract

Prior literature on adversarial attack methods has mainly focused on attacking with and defending against a single threat model, e.g., perturbations bounded in Lp ball. However, multiple threat models can be combined into composite perturbations. One such approach, composite adversarial attack (CAA), not only expands the perturbable space of the image, but also may be overlooked by current modes of robustness evaluation. This paper demonstrates how CAA's attack order affects the resulting image, and provides real-time inferences of different models, which will facilitate users' configuration of the parameters of the attack level and their rapid evaluation of model prediction. A leaderboard to benchmark adversarial robustness against CAA is also introduced.