SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing

TL;DR

SemanticAdv introduces a method to generate unrestricted adversarial examples by manipulating semantic attributes of images, revealing vulnerabilities of DNNs across various tasks and real-world services.

Contribution

We propose SemanticAdv, a novel algorithm that uses disentangled semantic factors to create targeted adversarial examples through semantic attribute manipulation.

Findings

Semantic adversarial examples can fool face verification and landmark detection.

High success rate in targeted attacks on real-world black-box services.

Semantic perturbations are effective across different image domains.

Abstract

Deep neural networks (DNNs) have achieved great success in various applications due to their strong expressive power. However, recent studies have shown that DNNs are vulnerable to adversarial examples which are manipulated instances targeting to mislead DNNs to make incorrect predictions. Currently, most such adversarial examples try to guarantee "subtle perturbation" by limiting the $L_p$ norm of the perturbation. In this paper, we aim to explore the impact of semantic manipulation on DNNs predictions by manipulating the semantic attributes of images and generate "unrestricted adversarial examples". In particular, we propose an algorithm \emph{SemanticAdv} which leverages disentangled semantic factors to generate adversarial perturbation by altering controlled semantic attributes to fool the learner towards various "adversarial" targets. We conduct extensive experiments to show that the semantic based adversarial examples can not only fool different learning tasks such as face verification and landmark detection, but also achieve high targeted attack success rate against \emph{real-world black-box} services such as Azure face verification service based on transferability. To further demonstrate the applicability of \emph{SemanticAdv} beyond face recognition domain, we also generate semantic perturbations on street-view images. Such adversarial examples with controlled semantic manipulation can shed light on further understanding about vulnerabilities of DNNs as well as potential defensive approaches.