FSPGD: Rethinking Black-box Attacks on Semantic Segmentation

TL;DR

FSPGD is a novel black-box attack method for semantic segmentation that improves transferability and attack success by leveraging intermediate layer features and a specialized loss function, outperforming existing methods.

Contribution

The paper introduces FSPGD, a new black-box attack approach that enhances transferability and attack effectiveness by utilizing feature similarity and spatial relationship disruptions.

Findings

FSPGD achieves state-of-the-art transferability on Pascal VOC 2012 and Cityscapes.

FSPGD outperforms existing attack methods in attack success rate.

The method effectively disrupts both local and contextual information in segmentation models.

Abstract

Transferability, the ability of adversarial examples crafted for one model to deceive other models, is crucial for black-box attacks. Despite advancements in attack methods for semantic segmentation, transferability remains limited, reducing their effectiveness in real-world applications. To address this, we introduce the Feature Similarity Projected Gradient Descent (FSPGD) attack, a novel black-box approach that enhances both attack performance and transferability. Unlike conventional segmentation attacks that rely on output predictions for gradient calculation, FSPGD computes gradients from intermediate layer features. Specifically, our method introduces a loss function that targets local information by comparing features between clean images and adversarial examples, while also disrupting contextual information by accounting for spatial relationships between objects. Experiments on Pascal VOC 2012 and Cityscapes datasets demonstrate that FSPGD achieves superior transferability and attack performance, establishing a new state-of-the-art benchmark. Code is available at https://github.com/KU-AIVS/FSPGD.