Semantic Adversarial Examples

TL;DR

This paper introduces 'Semantic Adversarial Examples' that are visually similar to original images but fool neural networks, using color-shifting transformations based on human shape bias, revealing new vulnerabilities.

Contribution

The paper proposes a novel class of adversarial examples that preserve semantic content while fooling models, using a shape bias-inspired color-shifting method.

Findings

VGG16 accuracy drops to 5.7% on adversarial color-shifted images.

Semantic adversarial examples are effective despite preserving the original object semantics.

The method leverages human shape bias to generate robust adversarial transformations.

Abstract

Deep neural networks are known to be vulnerable to adversarial examples, i.e., images that are maliciously perturbed to fool the model. Generating adversarial examples has been mostly limited to finding small perturbations that maximize the model prediction error. Such images, however, contain artificial perturbations that make them somewhat distinguishable from natural images. This property is used by several defense methods to counter adversarial examples by applying denoising filters or training the model to be robust to small perturbations. In this paper, we introduce a new class of adversarial examples, namely "Semantic Adversarial Examples," as images that are arbitrarily perturbed to fool the model, but in such a way that the modified image semantically represents the same object as the original image. We formulate the problem of generating such images as a constrained optimization problem and develop an adversarial transformation based on the shape bias property of human cognitive system. In our method, we generate adversarial images by first converting the RGB image into the HSV (Hue, Saturation and Value) color space and then randomly shifting the Hue and Saturation components, while keeping the Value component the same. Our experimental results on CIFAR10 dataset show that the accuracy of VGG16 network on adversarial color-shifted images is 5.7%.