What Learned Representations and Influence Functions Can Tell Us About Adversarial Examples

TL;DR

This paper adapts techniques from image processing to NLP for detecting adversarial examples, using learned representations, influence functions, and Mahalanobis distances, achieving state-of-the-art results and providing insights into adversarial subspaces.

Contribution

It introduces novel NLP-specific methods for characterizing adversarial subspaces, including a state-of-the-art detector based on influence functions and nearest neighbors.

Findings

Influence functions enhance adversarial detection in NLP.

The proposed detector outperforms several strong baselines.

Adversarial subspaces in NLP differ based on task type.

Abstract

Adversarial examples, deliberately crafted using small perturbations to fool deep neural networks, were first studied in image processing and more recently in NLP. While approaches to detecting adversarial examples in NLP have largely relied on search over input perturbations, image processing has seen a range of techniques that aim to characterise adversarial subspaces over the learned representations. In this paper, we adapt two such approaches to NLP, one based on nearest neighbors and influence functions and one on Mahalanobis distances. The former in particular produces a state-of-the-art detector when compared against several strong baselines; moreover, the novel use of influence functions provides insight into how the nature of adversarial example subspaces in NLP relate to those in image processing, and also how they differ depending on the kind of NLP task.