Exploring the Landscape of Spatial Robustness

TL;DR

This paper investigates neural network vulnerabilities to natural spatial transformations like rotations and translations, demonstrating that robust optimization and test-time aggregation can improve robustness, and highlighting fundamental differences from p-norm adversarial attacks.

Contribution

It provides a comprehensive analysis of spatial robustness, introduces methods to enhance it, and reveals that first-order attacks are ineffective in this setting, marking a new research direction.

Findings

Data augmentation offers limited robustness improvements.

Robust optimization and test-time aggregation significantly enhance spatial robustness.

First-order methods cannot reliably find worst-case spatial perturbations.

Abstract

The study of adversarial robustness has so far largely focused on perturbations bound in p-norms. However, state-of-the-art models turn out to be also vulnerable to other, more natural classes of perturbations such as translations and rotations. In this work, we thoroughly investigate the vulnerability of neural network--based classifiers to rotations and translations. While data augmentation offers relatively small robustness, we use ideas from robust optimization and test-time input aggregation to significantly improve robustness. Finally we find that, in contrast to the p-norm case, first-order methods cannot reliably find worst-case perturbations. This highlights spatial robustness as a fundamentally different setting requiring additional study. Code available at https://github.com/MadryLab/adversarial_spatial and https://github.com/MadryLab/spatial-pytorch.