Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles

TL;DR

This paper introduces a visual framework to analyze how neural networks perceive adversarial examples differently from regular data, aiding in identifying vulnerabilities and guiding improvements in model robustness.

Contribution

It presents a novel visualization method to investigate neural network responses to adversarial attacks, revealing differences in perception and aiding in vulnerability analysis.

Findings

Visual framework reveals perception differences between regular and adversarial data

Helps identify exploited vulnerabilities in neural network models

Guides improvements in training and architecture for robustness

Abstract

As neural networks become the tool of choice to solve an increasing variety of problems in our society, adversarial attacks become critical. The possibility of generating data instances deliberately designed to fool a network's analysis can have disastrous consequences. Recent work has shown that commonly used methods for model training often result in fragile abstract representations that are particularly vulnerable to such attacks. This paper presents a visual framework to investigate neural network models subjected to adversarial examples, revealing how models' perception of the adversarial data differs from regular data instances and their relationships with class perception. Through different use cases, we show how observing these elements can quickly pinpoint exploited areas in a model, allowing further study of vulnerable features in input data and serving as a guide to improving model training and architecture.