Designing DNNs for a trade-off between robustness and processing performance in embedded devices

TL;DR

This paper explores how the choice of activation functions in DNNs affects their robustness to soft errors, especially in embedded safety-critical systems, while considering trade-offs with accuracy and efficiency.

Contribution

It introduces the use of bounded activation functions to enhance DNN robustness against parameter perturbations in embedded applications.

Findings

Bounded activation functions improve robustness to soft errors.

Trade-offs between robustness, accuracy, and computational complexity are analyzed.

Experimental validation on hyperspectral image segmentation models.

Abstract

Machine learning-based embedded systems employed in safety-critical applications such as aerospace and autonomous driving need to be robust against perturbations produced by soft errors. Soft errors are an increasing concern in modern digital processors since smaller transistor geometries and lower voltages give electronic devices a higher sensitivity to background radiation. The resilience of deep neural network (DNN) models to perturbations in their parameters is determined, to a large extent, by the structure of the model itself, and also by the selected numerical representation and used arithmetic precision. When compression techniques such as model pruning and model quantization are applied to reduce memory footprint and computational complexity for deployment, both model structure and numerical representation are modified and thus, soft error robustness also changes. In this sense, although the choice of activation functions (AFs) in DNN models is frequently ignored, it conditions not only their accuracy and trainability, but also compressibility rates and numerical robustness. This paper investigates the suitability of using bounded AFs to improve model robustness against DNN parameter perturbations, assessing at the same time the impact of this choice on deployment in terms of model accuracy, compressibility, and computational burden. In particular, we analyze encoder-decoder fully convolutional models aimed at performing semantic segmentation tasks on hyperspectral images for scene understanding in autonomous driving. Deployment characterization is performed experimentally on an AMD-Xilinx's KV260 SoM.