ActGraph: Prioritization of Test Cases Based on Deep Neural Network Activation Graph

TL;DR

ActGraph introduces a novel activation graph-based approach for prioritizing test cases in DNN testing, improving efficiency and effectiveness across various scenarios by leveraging high-order neuron features.

Contribution

This paper proposes ActGraph, a new test case prioritization method using activation graphs that addresses scenario limitations and reduces time complexity without mutation operations.

Findings

Achieves ~1.40 RAUC-100 improvement in diverse scenarios.

Runs approximately 50 times faster than existing methods.

Demonstrates effectiveness across three datasets and four models.

Abstract

Widespread applications of deep neural networks (DNNs) benefit from DNN testing to guarantee their quality. In the DNN testing, numerous test cases are fed into the model to explore potential vulnerabilities, but they require expensive manual cost to check the label. Therefore, test case prioritization is proposed to solve the problem of labeling cost, e.g., activation-based and mutation-based prioritization methods. However, most of them suffer from limited scenarios (i.e. high confidence adversarial or false positive cases) and high time complexity. To address these challenges, we propose the concept of the activation graph from the perspective of the spatial relationship of neurons. We observe that the activation graph of cases that triggers the models' misbehavior significantly differs from that of normal cases. Motivated by it, we design a test case prioritization method based on the activation graph, ActGraph, by extracting the high-order node features of the activation graph for prioritization. ActGraph explains the difference between the test cases to solve the problem of scenario limitation. Without mutation operations, ActGraph is easy to implement, leading to lower time complexity. Extensive experiments on three datasets and four models demonstrate that ActGraph has the following key characteristics. (i) Effectiveness and generalizability: ActGraph shows competitive performance in all of the natural, adversarial and mixed scenarios, especially in RAUC-100 improvement (~1.40). (ii) Efficiency: ActGraph does not use complex mutation operations and runs in less time (~1/50) than the state-of-the-art method.