Evaluating Defensive Distillation For Defending Text Processing Neural Networks Against Adversarial Examples

TL;DR

This paper assesses the effectiveness of defensive distillation, a technique to defend neural networks against adversarial examples, specifically in text classification, and finds it offers minimal robustness improvements.

Contribution

First evaluation of defensive distillation's impact on text classification neural networks and its effect on adversarial transferability.

Findings

Minimal impact on robustness against adversarial text examples

Does not significantly reduce transferability of adversarial examples

Limited effectiveness compared to image domain applications

Abstract

Adversarial examples are artificially modified input samples which lead to misclassifications, while not being detectable by humans. These adversarial examples are a challenge for many tasks such as image and text classification, especially as research shows that many adversarial examples are transferable between different classifiers. In this work, we evaluate the performance of a popular defensive strategy for adversarial examples called defensive distillation, which can be successful in hardening neural networks against adversarial examples in the image domain. However, instead of applying defensive distillation to networks for image classification, we examine, for the first time, its performance on text classification tasks and also evaluate its effect on the transferability of adversarial text examples. Our results indicate that defensive distillation only has a minimal impact on text classifying neural networks and does neither help with increasing their robustness against adversarial examples nor prevent the transferability of adversarial examples between neural networks.