On Functional Test Generation for Deep Neural Network IPs

TL;DR

This paper presents a gradient-based method for generating functional test cases for third-party DNN IP cores, enabling validation of their functionality solely through output checks without revealing internal parameters.

Contribution

It introduces a novel approach to generate test cases by selecting training samples and applying gradients, addressing the challenge of testing complex DNN IPs securely.

Findings

Effective test case generation activates many parameters.

Validation achieved through output comparison alone.

Method demonstrates high efficacy in experiments.

Abstract

Machine learning systems based on deep neural networks (DNNs) produce state-of-the-art results in many applications. Considering the large amount of training data and know-how required to generate the network, it is more practical to use third-party DNN intellectual property (IP) cores for many designs. No doubt to say, it is essential for DNN IP vendors to provide test cases for functional validation without leaking their parameters to IP users. To satisfy this requirement, we propose to effectively generate test cases that activate parameters as many as possible and propagate their perturbations to outputs. Then the functionality of DNN IPs can be validated by only checking their outputs. However, it is difficult considering large numbers of parameters and highly non-linearity of DNNs. In this paper, we tackle this problem by judiciously selecting samples from the DNN training set and applying a gradient-based method to generate new test cases. Experimental results demonstrate the efficacy of our proposed solution.