Entropy-Based Modeling for Estimating Soft Errors Impact on Binarized Neural Network Inference

TL;DR

This paper introduces entropy-based statistical models to evaluate the impact of radiation-induced soft errors on binarized neural network inference, focusing on layer-specific vulnerability assessment.

Contribution

It presents novel entropy-based models for estimating soft error effects on neural network layers, aiding in assessing error resilience for safety-critical applications.

Findings

Weights and activations are more vulnerable in early layers.

Soft errors can significantly degrade inference accuracy.

Layer-specific susceptibility varies across the network.

Abstract

Over past years, the easy accessibility to the large scale datasets has significantly shifted the paradigm for developing highly accurate prediction models that are driven from Neural Network (NN). These models can be potentially impacted by the radiation-induced transient faults that might lead to the gradual downgrade of the long-running expected NN inference accelerator. The crucial observation from our rigorous vulnerability assessment on the NN inference accelerator demonstrates that the weights and activation functions are unevenly susceptible to both single-event upset (SEU) and multi-bit upset (MBU), especially in the first five layers of our selected convolution neural network. In this paper, we present the relatively-accurate statistical models to delineate the impact of both undertaken SEU and MBU across layers and per each layer of the selected NN. These models can be used for evaluating the error-resiliency magnitude of NN topology before adopting them in the safety-critical applications.