# Perceptual Evaluation of Adversarial Attacks for CNN-based Image Classification

## TL;DR

This paper introduces a new database and evaluation of perceptual metrics for adversarial examples in CNN image classification, highlighting the inadequacy of traditional $L_{p}$ norms for human-aligned similarity assessment.

## Contribution

It presents a novel database for visual fidelity of adversarial examples and evaluates fifteen state-of-the-art perceptual metrics against human judgment.

## Key findings

- Traditional $L_{p}$ norms do not align with human perception.
- Fifteen perceptual metrics were assessed for effectiveness.
- The database and scores are publicly available for future research.

## Abstract

Deep neural networks (DNNs) have recently achieved state-of-the-art performance and provide significant progress in many machine learning tasks, such as image classification, speech processing, natural language processing, etc. However, recent studies have shown that DNNs are vulnerable to adversarial attacks. For instance, in the image classification domain, adding small imperceptible perturbations to the input image is sufficient to fool the DNN and to cause misclassification. The perturbed image, called \textit{adversarial example}, should be visually as close as possible to the original image. However, all the works proposed in the literature for generating adversarial examples have used the $L_{p}$ norms ($L_{0}$, $L_{2}$ and $L_{\infty}$) as distance metrics to quantify the similarity between the original image and the adversarial example. Nonetheless, the $L_{p}$ norms do not correlate with human judgment, making them not suitable to reliably assess the perceptual similarity/fidelity of adversarial examples. In this paper, we present a database for visual fidelity assessment of adversarial examples. We describe the creation of the database and evaluate the performance of fifteen state-of-the-art full-reference (FR) image fidelity assessment metrics that could substitute $L_{p}$ norms. The database as well as subjective scores are publicly available to help designing new metrics for adversarial examples and to facilitate future research works.