On Hardening DNNs against Noisy Computations
Xiao Wang, Hendrik Borras, Bernhard Klein, Holger Fr\"oning

TL;DR
This paper explores methods to improve the robustness of deep neural networks against noisy analog computations, finding that noisy training generally outperforms quantization-aware training in maintaining accuracy under noise.
Contribution
It compares quantization-aware training and noisy training for enhancing neural network robustness against analog computation noise, highlighting the superiority of noisy training.
Findings
Noisy training outperforms quantization-aware training in robustness.
Quantization-aware training with constant scaling improves noise tolerance.
Robustness gains are consistent across various neural network architectures.
Abstract
The success of deep learning has sparked significant interest in designing computer hardware optimized for the high computational demands of neural network inference. As further miniaturization of digital CMOS processors becomes increasingly challenging, alternative computing paradigms, such as analog computing, are gaining consideration. Particularly for compute-intensive tasks such as matrix multiplication, analog computing presents a promising alternative due to its potential for significantly higher energy efficiency compared to conventional digital technology. However, analog computations are inherently noisy, which makes it challenging to maintain high accuracy on deep neural networks. This work investigates the effectiveness of training neural networks with quantization to increase the robustness against noise. Experimental results across various network architectures show that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Advanced Neural Network Applications · Brain Tumor Detection and Classification
