Automated design of error-resilient and hardware-efficient deep neural networks

TL;DR

This paper presents an evolutionary optimization method for automatically designing deep neural networks that are both hardware-efficient and resilient to errors, crucial for safety-critical and mobile applications.

Contribution

It introduces a fast evaluation approach for DNN architectures focusing on error resilience and efficiency, enabling automated multi-objective optimization.

Findings

Strong correlation between predicted and actual error resilience.

Quantization schemes significantly impact error resilience.

Optimized architectures improve hardware efficiency and fault tolerance.

Abstract

Applying deep neural networks (DNNs) in mobile and safety-critical systems, such as autonomous vehicles, demands a reliable and efficient execution on hardware. Optimized dedicated hardware accelerators are being developed to achieve this. However, the design of efficient and reliable hardware has become increasingly difficult, due to the increased complexity of modern integrated circuit technology and its sensitivity against hardware faults, such as random bit-flips. It is thus desirable to exploit optimization potential for error resilience and efficiency also at the algorithmic side, e.g., by optimizing the architecture of the DNN. Since there are numerous design choices for the architecture of DNNs, with partially opposing effects on the preferred characteristics (such as small error rates at low latency), multi-objective optimization strategies are necessary. In this paper, we develop an evolutionary optimization technique for the automated design of hardware-optimized DNN architectures. For this purpose, we derive a set of easily computable objective functions, which enable the fast evaluation of DNN architectures with respect to their hardware efficiency and error resilience solely based on the network topology. We observe a strong correlation between predicted error resilience and actual measurements obtained from fault injection simulations. Furthermore, we analyze two different quantization schemes for efficient DNN computation and find significant differences regarding their effect on error resilience.