Semantically Adversarial Learnable Filters

TL;DR

This paper introduces a novel adversarial framework that crafts perturbations considering image content and label semantics, effectively fooling classifiers while maintaining image filter characteristics.

Contribution

It proposes a multi-task neural network framework combining structure and semantic losses to generate targeted, semantically-aware adversarial perturbations for image classifiers.

Findings

High success rate in fooling classifiers

Robustness and transferability demonstrated across multiple models

Effective against various image filters like detail enhancement and gamma correction

Abstract

We present an adversarial framework to craft perturbations that mislead classifiers by accounting for the image content and the semantics of the labels. The proposed framework combines a structure loss and a semantic adversarial loss in a multi-task objective function to train a fully convolutional neural network. The structure loss helps generate perturbations whose type and magnitude are defined by a target image processing filter. The semantic adversarial loss considers groups of (semantic) labels to craft perturbations that prevent the filtered image {from} being classified with a label in the same group. We validate our framework with three different target filters, namely detail enhancement, log transformation and gamma correction filters; and evaluate the adversarially filtered images against three classifiers, ResNet50, ResNet18 and AlexNet, pre-trained on ImageNet. We show that the proposed framework generates filtered images with a high success rate, robustness, and transferability to unseen classifiers. We also discuss objective and subjective evaluations of the adversarial perturbations.