DeepDyve: Dynamic Verification for Deep Neural Networks

TL;DR

DeepDyve introduces a lightweight, dynamic verification method using smaller neural networks to enhance fault tolerance in DNN systems, significantly reducing risks with minimal overhead.

Contribution

The paper presents a novel fault-tolerant approach for DNNs using pre-trained smaller networks for dynamic verification, addressing faults within the system itself.

Findings

DeepDyve reduces 90% of risks in DNN systems.

The method achieves around 10% overhead.

Efficient architecture and task exploration optimize risk/overhead trade-off.

Abstract

Deep neural networks (DNNs) have become one of the enabling technologies in many safety-critical applications, e.g., autonomous driving and medical image analysis. DNN systems, however, suffer from various kinds of threats, such as adversarial example attacks and fault injection attacks. While there are many defense methods proposed against maliciously crafted inputs, solutions against faults presented in the DNN system itself (e.g., parameters and calculations) are far less explored. In this paper, we develop a novel lightweight fault-tolerant solution for DNN-based systems, namely DeepDyve, which employs pre-trained neural networks that are far simpler and smaller than the original DNN for dynamic verification. The key to enabling such lightweight checking is that the smaller neural network only needs to produce approximate results for the initial task without sacrificing fault coverage much. We develop efficient and effective architecture and task exploration techniques to achieve optimized risk/overhead trade-off in DeepDyve. Experimental results show that DeepDyve can reduce 90% of the risks at around 10% overhead.