Indirect Local Attacks for Context-aware Semantic Segmentation Networks

TL;DR

This paper reveals that advanced semantic segmentation networks are vulnerable to indirect local attacks confined to small image regions, and proposes strategies for such attacks and detection methods.

Contribution

Introduces novel indirect local attack strategies and detection techniques for context-aware semantic segmentation networks, highlighting their vulnerability due to reliance on large contextual information.

Findings

More accurate networks are more sensitive to local attacks.

Proposed attack strategies include adaptive and universal local attacks.

Detection methods can identify fooled regions at pixel level.

Abstract

Recently, deep networks have achieved impressive semantic segmentation performance, in particular thanks to their use of larger contextual information. In this paper, we show that the resulting networks are sensitive not only to global attacks, where perturbations affect the entire input image, but also to indirect local attacks where perturbations are confined to a small image region that does not overlap with the area that we aim to fool. To this end, we introduce several indirect attack strategies, including adaptive local attacks, aiming to find the best image location to perturb, and universal local attacks. Furthermore, we propose attack detection techniques both for the global image level and to obtain a pixel-wise localization of the fooled regions. Our results are unsettling: Because they exploit a larger context, more accurate semantic segmentation networks are more sensitive to indirect local attacks.