A Noise-Sensitivity-Analysis-Based Test Prioritization Technique for Deep Neural Networks

TL;DR

This paper introduces a test prioritization method for deep neural networks based on analyzing the noise sensitivity of examples, aiming to improve adversarial example generation and robustness evaluation.

Contribution

It proposes a novel noise-sensitivity-analysis-based technique to select examples according to their noise sensitivity, enhancing adversarial testing strategies.

Findings

Examples vary in noise sensitivity across datasets and models

The method effectively identifies noise-sensitive examples

It improves the efficiency of adversarial example generation

Abstract

Deep neural networks (DNNs) have been widely used in the fields such as natural language processing, computer vision and image recognition. But several studies have been shown that deep neural networks can be easily fooled by artificial examples with some perturbations, which are widely known as adversarial examples. Adversarial examples can be used to attack deep neural networks or to improve the robustness of deep neural networks. A common way of generating adversarial examples is to first generate some noises and then add them into original examples. In practice, different examples have different noise-sensitive. To generate an effective adversarial example, it may be necessary to add a lot of noise to low noise-sensitive example, which may make the adversarial example meaningless. In this paper, we propose a noise-sensitivity-analysis-based test prioritization technique to pick out examples by their noise sensitivity. We construct an experiment to validate our approach on four image sets and two DNN models, which shows that examples are sensitive to noise and our method can effectively pick out examples by their noise sensitivity.