# Applying Reinforcement Learning to Protect Deep Neural Networks from Soft Errors

## TL;DR

This paper introduces a reinforcement learning approach to protect deep neural networks from soft errors by identifying and protecting vulnerable bits, showing better performance than traditional methods.

## Contribution

The novel contribution is a reinforcement learning-based method for dynamically identifying and protecting vulnerable bits in deep neural networks.

## Key findings

- The proposed method achieves a performance gain of at least 10% to 15% over baseline methods.
- The reinforcement learning approach dynamically and efficiently protects vulnerable bits compared to traditional schemes.

## Abstract

With the advance of Artificial Intelligence, Deep Neural Networks are widely employed in various sensor-based systems to analyze operational conditions. However, due to the inherently nondeterministic and probabilistic natures of neural networks, the assurance of overall system performance could become a challenging task. In particular, soft errors could weaken the robustness of such networks and thereby threaten the system’s safety. Conventional fault-tolerant techniques by means of hardware redundancy and software correction mechanisms often involve a tricky trade-off between effectiveness and scalability in addressing the extensive design space of Deep Neural Networks. In this work, we propose a Reinforcement-Learning-based approach to protect neural networks from soft errors by addressing and identifying the vulnerable bits. The approach consists of three key steps: (1) analyzing layer-wise resiliency of Deep Neural Networks by a fault injection simulation; (2) generating layer-wise bit masks by a Reinforcement-Learning-based agent to reveal the vulnerable bits and to protect against them; and (3) synthesizing and deploying bit masks across the network with guaranteed operation efficiency by adopting transfer learning. As a case study, we select several existing neural networks to test and validate the design. The performance of the proposed approach is compared with the performance of other baseline methods, including Hamming code and the Most Significant Bits protection schemes. The results indicate that the proposed method exhibits a significant improvement. Specifically, we observe that the proposed method achieves a significant performance gain of at least 10% to 15% over on the test network. The results indicate that the proposed method dynamically and efficiently protects the vulnerable bits compared with the baseline methods.