GGT: Graph-Guided Testing for Adversarial Sample Detection of Deep Neural Network

TL;DR

This paper introduces Graph-Guided Testing (GGT), a novel method for detecting adversarial samples in deep neural networks that is more efficient and effective than previous approaches like Model Mutation Testing.

Contribution

GGT uses graph-guided model pruning to create diverse, smaller models for adversarial detection, improving over existing methods in efficiency and effectiveness.

Findings

GGT achieves higher detection accuracy than MMT.

GGT models are about 5% the size of MMT models.

GGT demonstrates better efficiency and effectiveness on CIFAR10 and SVHN.

Abstract

Deep Neural Networks (DNN) are known to be vulnerable to adversarial samples, the detection of which is crucial for the wide application of these DNN models. Recently, a number of deep testing methods in software engineering were proposed to find the vulnerability of DNN systems, and one of them, i.e., Model Mutation Testing (MMT), was used to successfully detect various adversarial samples generated by different kinds of adversarial attacks. However, the mutated models in MMT are always huge in number (e.g., over 100 models) and lack diversity (e.g., can be easily circumvented by high-confidence adversarial samples), which makes it less efficient in real applications and less effective in detecting high-confidence adversarial samples. In this study, we propose Graph-Guided Testing (GGT) for adversarial sample detection to overcome these aforementioned challenges. GGT generates pruned models with the guide of graph characteristics, each of them has only about 5% parameters of the mutated model in MMT, and graph guided models have higher diversity. The experiments on CIFAR10 and SVHN validate that GGT performs much better than MMT with respect to both effectiveness and efficiency.