On the Feasibility and Generality of Patch-based Adversarial Attacks on Semantic Segmentation Problems

TL;DR

This paper investigates the feasibility of patch-based adversarial attacks on semantic segmentation networks, showing they are limited in scope and cannot generate arbitrary outputs in practical scenarios.

Contribution

It demonstrates through case studies that patch-based attacks are less general and more spatially limited than previously assumed, especially in real-world applications.

Findings

Patch-based attacks can alter segmentation outputs in specific cases.

The number of possible output maps from such attacks is limited.

Most patch-based attacks are not capable of generating arbitrary outputs in practice.

Abstract

Deep neural networks were applied with success in a myriad of applications, but in safety critical use cases adversarial attacks still pose a significant threat. These attacks were demonstrated on various classification and detection tasks and are usually considered general in a sense that arbitrary network outputs can be generated by them. In this paper we will demonstrate through simple case studies both in simulation and in real-life, that patch based attacks can be utilised to alter the output of segmentation networks. Through a few examples and the investigation of network complexity, we will also demonstrate that the number of possible output maps which can be generated via patch-based attacks of a given size is typically smaller than the area they effect or areas which should be attacked in case of practical applications. We will prove that based on these results most patch-based attacks cannot be general in practice, namely they can not generate arbitrary output maps or if they could, they are spatially limited and this limit is significantly smaller than the receptive field of the patches.