Efficient Noise Mitigation for Enhancing Inference Accuracy in DNNs on Mixed-Signal Accelerators

TL;DR

This paper introduces a noise mitigation framework with denoising blocks and an insertion algorithm to improve DNN inference accuracy on mixed-signal accelerators, significantly reducing accuracy loss caused by analog component variations.

Contribution

It presents a novel approach combining denoising blocks, optimal insertion algorithms, and efficient architecture design to enhance robustness of DNNs on mixed-signal hardware.

Findings

Accuracy drop reduced from 31.7% to 1.15%.

Achieved this with only 2.03% parameter overhead.

Effective on ImageNet and CIFAR-10 datasets.

Abstract

In this paper, we propose a framework to enhance the robustness of the neural models by mitigating the effects of process-induced and aging-related variations of analog computing components on the accuracy of the analog neural networks. We model these variations as the noise affecting the precision of the activations and introduce a denoising block inserted between selected layers of a pre-trained model. We demonstrate that training the denoising block significantly increases the model's robustness against various noise levels. To minimize the overhead associated with adding these blocks, we present an exploration algorithm to identify optimal insertion points for the denoising blocks. Additionally, we propose a specialized architecture to efficiently execute the denoising blocks, which can be integrated into mixed-signal accelerators. We evaluate the effectiveness of our approach using Deep Neural Network (DNN) models trained on the ImageNet and CIFAR-10 datasets. The results show that on average, by accepting 2.03% parameter count overhead, the accuracy drop due to the variations reduces from 31.7% to 1.15%.