DeepPerform: An Efficient Approach for Performance Testing of Resource-Constrained Neural Networks

TL;DR

DeepPerform is a scalable method that efficiently generates test inputs to identify input-dependent performance bottlenecks in resource-constrained neural networks, improving testing effectiveness and efficiency.

Contribution

It introduces a novel optimization-based approach to generate performance test samples for AdNNs, addressing a gap in existing correctness-focused testing methods.

Findings

DeepPerform detects inputs causing up to 552% increase in FLOPs.

It significantly reduces test input generation time to 6-10 milliseconds.

The approach outperforms baseline methods in efficiency and effectiveness.

Abstract

Today, an increasing number of Adaptive Deep Neural Networks (AdNNs) are being used on resource-constrained embedded devices. We observe that, similar to traditional software, redundant computation exists in AdNNs, resulting in considerable performance degradation. The performance degradation is dependent on the input and is referred to as input-dependent performance bottlenecks (IDPBs). To ensure an AdNN satisfies the performance requirements of resource-constrained applications, it is essential to conduct performance testing to detect IDPBs in the AdNN. Existing neural network testing methods are primarily concerned with correctness testing, which does not involve performance testing. To fill this gap, we propose DeepPerform, a scalable approach to generate test samples to detect the IDPBs in AdNNs. We first demonstrate how the problem of generating performance test samples detecting IDPBs can be formulated as an optimization problem. Following that, we demonstrate how DeepPerform efficiently handles the optimization problem by learning and estimating the distribution of AdNNs' computational consumption. We evaluate DeepPerform on three widely used datasets against five popular AdNN models. The results show that DeepPerform generates test samples that cause more severe performance degradation (FLOPs: increase up to 552\%). Furthermore, DeepPerform is substantially more efficient than the baseline methods in generating test inputs(runtime overhead: only 6-10 milliseconds).