Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples

TL;DR

This paper introduces an interpretability-based method for detecting adversarial samples in face recognition models by analyzing neuron-attribute relationships, achieving high detection accuracy across multiple attack types.

Contribution

It proposes a novel bi-directional inference approach linking attributes and neurons to identify adversarial inputs, improving detection accuracy over existing methods.

Findings

94% detection accuracy on 7 attack types

9.91% false positive rate on benign inputs

outperforms feature squeezing technique

Abstract

Adversarial sample attacks perturb benign inputs to induce DNN misbehaviors. Recent research has demonstrated the widespread presence and the devastating consequences of such attacks. Existing defense techniques either assume prior knowledge of specific attacks or may not work well on complex models due to their underlying assumptions. We argue that adversarial sample attacks are deeply entangled with interpretability of DNN models: while classification results on benign inputs can be reasoned based on the human perceptible features/attributes, results on adversarial samples can hardly be explained. Therefore, we propose a novel adversarial sample detection technique for face recognition models, based on interpretability. It features a novel bi-directional correspondence inference between attributes and internal neurons to identify neurons critical for individual attributes. The activation values of critical neurons are enhanced to amplify the reasoning part of the computation and the values of other neurons are weakened to suppress the uninterpretable part. The classification results after such transformation are compared with those of the original model to detect adversaries. Results show that our technique can achieve 94% detection accuracy for 7 different kinds of attacks with 9.91% false positives on benign inputs. In contrast, a state-of-the-art feature squeezing technique can only achieve 55% accuracy with 23.3% false positives.