Towards Deep Neural Network Architectures Robust to Adversarial Examples

TL;DR

This paper investigates the vulnerability of deep neural networks to adversarial examples and proposes a new training method called Deep Contractive Network to enhance robustness without sacrificing accuracy.

Contribution

It introduces the Deep Contractive Network with an end-to-end training procedure that incorporates a smoothness penalty, improving adversarial robustness.

Findings

Denoising autoencoders can remove much adversarial noise

Stacked DAE and DNN can still be attacked with smaller distortions

Deep Contractive Network increases robustness without significant performance loss

Abstract

Recent work has shown deep neural networks (DNNs) to be highly susceptible to well-designed, small perturbations at the input layer, or so-called adversarial examples. Taking images as an example, such distortions are often imperceptible, but can result in 100% mis-classification for a state of the art DNN. We study the structure of adversarial examples and explore network topology, pre-processing and training strategies to improve the robustness of DNNs. We perform various experiments to assess the removability of adversarial examples by corrupting with additional noise and pre-processing with denoising autoencoders (DAEs). We find that DAEs can remove substantial amounts of the adversarial noise. How- ever, when stacking the DAE with the original DNN, the resulting network can again be attacked by new adversarial examples with even smaller distortion. As a solution, we propose Deep Contractive Network, a model with a new end-to-end training procedure that includes a smoothness penalty inspired by the contractive autoencoder (CAE). This increases the network robustness to adversarial examples, without a significant performance penalty.