Investigating Human-Identifiable Features Hidden in Adversarial Perturbations

TL;DR

This paper investigates human-identifiable features in adversarial perturbations across multiple attack algorithms and datasets, revealing effects like masking and generation, and providing insights into transferability and model interpretability.

Contribution

It uncovers human-identifiable features in adversarial perturbations and distinguishes effects in targeted versus untargeted attacks, advancing understanding of attack mechanisms.

Findings

Identification of human-identifiable features in adversarial perturbations

Distinct masking and generation effects in untargeted and targeted attacks

Perturbations show similarity across different attack algorithms and models

Abstract

Neural networks perform exceedingly well across various machine learning tasks but are not immune to adversarial perturbations. This vulnerability has implications for real-world applications. While much research has been conducted, the underlying reasons why neural networks fall prey to adversarial attacks are not yet fully understood. Central to our study, which explores up to five attack algorithms across three datasets, is the identification of human-identifiable features in adversarial perturbations. Additionally, we uncover two distinct effects manifesting within human-identifiable features. Specifically, the masking effect is prominent in untargeted attacks, while the generation effect is more common in targeted attacks. Using pixel-level annotations, we extract such features and demonstrate their ability to compromise target models. In addition, our findings indicate a notable extent of similarity in perturbations across different attack algorithms when averaged over multiple models. This work also provides insights into phenomena associated with adversarial perturbations, such as transferability and model interpretability. Our study contributes to a deeper understanding of the underlying mechanisms behind adversarial attacks and offers insights for the development of more resilient defense strategies for neural networks.