On Evaluating the Adversarial Robustness of Semantic Segmentation Models

TL;DR

This paper emphasizes the importance of comprehensive evaluation of adversarial robustness in semantic segmentation models, proposing new attack methods and demonstrating that many models previously deemed robust are vulnerable, while simple adversarial training can improve robustness.

Contribution

It introduces a set of gradient-based iterative attacks for semantic segmentation and highlights the necessity of using only adversarial examples during training for robustness.

Findings

Many models claimed to be robust are actually vulnerable.

Simple adversarial training improves robustness under strong attacks.

Using only adversarial examples during training is key to robustness.

Abstract

Achieving robustness against adversarial input perturbation is an important and intriguing problem in machine learning. In the area of semantic image segmentation, a number of adversarial training approaches have been proposed as a defense against adversarial perturbation, but the methodology of evaluating the robustness of the models is still lacking, compared to image classification. Here, we demonstrate that, just like in image classification, it is important to evaluate the models over several different and hard attacks. We propose a set of gradient based iterative attacks and show that it is essential to perform a large number of iterations. We include attacks against the internal representations of the models as well. We apply two types of attacks: maximizing the error with a bounded perturbation, and minimizing the perturbation for a given level of error. Using this set of attacks, we show for the first time that a number of models in previous work that are claimed to be robust are in fact not robust at all. We then evaluate simple adversarial training algorithms that produce reasonably robust models even under our set of strong attacks. Our results indicate that a key design decision to achieve any robustness is to use only adversarial examples during training. However, this introduces a trade-off between robustness and accuracy.