Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation

TL;DR

This paper investigates how spatial consistency information in semantic segmentation can be used to detect adversarial examples, revealing their limited transferability and providing insights for improved defenses against attacks.

Contribution

It introduces a novel approach to characterize and detect adversarial examples in semantic segmentation using spatial context information, highlighting their non-transferability across models.

Findings

Spatial consistency can robustly detect adversarial examples.

Adversarial examples in segmentation barely transfer between models.

Insights into vulnerabilities of DNNs in semantic segmentation.

Abstract

Deep Neural Networks (DNNs) have been widely applied in various recognition tasks. However, recently DNNs have been shown to be vulnerable against adversarial examples, which can mislead DNNs to make arbitrary incorrect predictions. While adversarial examples are well studied in classification tasks, other learning problems may have different properties. For instance, semantic segmentation requires additional components such as dilated convolutions and multiscale processing. In this paper, we aim to characterize adversarial examples based on spatial context information in semantic segmentation. We observe that spatial consistency information can be potentially leveraged to detect adversarial examples robustly even when a strong adaptive attacker has access to the model and detection strategies. We also show that adversarial examples based on attacks considered within the paper barely transfer among models, even though transferability is common in classification. Our observations shed new light on developing adversarial attacks and defenses to better understand the vulnerabilities of DNNs.