Is current research on adversarial robustness addressing the right problem?

TL;DR

Current adversarial robustness research has provided insights but remains incomplete; a broader, more human-inspired approach to model design and problem formulation is needed to truly address vulnerabilities.

Contribution

The paper critiques existing problem formulations and advocates for exploring new model classes and broader robustness criteria inspired by human vision.

Findings

Current formulations focus on imperceptible perturbations.

Existing models are limited in expressiveness for robustness.

Broader robustness objectives may lead to better solutions.

Abstract

Short answer: Yes, Long answer: No! Indeed, research on adversarial robustness has led to invaluable insights helping us understand and explore different aspects of the problem. Many attacks and defenses have been proposed over the last couple of years. The problem, however, remains largely unsolved and poorly understood. Here, I argue that the current formulation of the problem serves short term goals, and needs to be revised for us to achieve bigger gains. Specifically, the bound on perturbation has created a somewhat contrived setting and needs to be relaxed. This has misled us to focus on model classes that are not expressive enough to begin with. Instead, inspired by human vision and the fact that we rely more on robust features such as shape, vertices, and foreground objects than non-robust features such as texture, efforts should be steered towards looking for significantly different classes of models. Maybe instead of narrowing down on imperceptible adversarial perturbations, we should attack a more general problem which is finding architectures that are simultaneously robust to perceptible perturbations, geometric transformations (e.g. rotation, scaling), image distortions (lighting, blur), and more (e.g. occlusion, shadow). Only then we may be able to solve the problem of adversarial vulnerability.