A Blueprint for Precise and Fault-Tolerant Analog Neural Networks

TL;DR

This paper proposes an RNS-based approach for analog neural networks that achieves high accuracy with lower precision data converters, significantly reducing energy consumption while maintaining performance and introducing fault tolerance.

Contribution

It introduces an RNS-based method for high-precision analog neural network computation, enabling high accuracy with lower precision converters and fault-tolerant dataflow.

Findings

Achieves ≥99% FP32 accuracy with 6-bit data converters.

Reduces energy consumption by orders of magnitude.

Extends approach to efficient DNN training with 7-bit arithmetic.

Abstract

Analog computing has reemerged as a promising avenue for accelerating deep neural networks (DNNs) due to its potential to overcome the energy efficiency and scalability challenges posed by traditional digital architectures. However, achieving high precision and DNN accuracy using such technologies is challenging, as high-precision data converters are costly and impractical. In this paper, we address this challenge by using the residue number system (RNS). RNS allows composing high-precision operations from multiple low-precision operations, thereby eliminating the information loss caused by the limited precision of the data converters. Our study demonstrates that analog accelerators utilizing the RNS-based approach can achieve ${\geq}99\%$ of FP32 accuracy for state-of-the-art DNN inference using data converters with only $6$-bit precision whereas a conventional analog core requires more than $8$-bit precision to achieve the same accuracy in the same DNNs. The reduced precision requirements imply that using RNS can reduce the energy consumption of analog accelerators by several orders of magnitude while maintaining the same throughput and precision. Our study extends this approach to DNN training, where we can efficiently train DNNs using $7$-bit integer arithmetic while achieving accuracy comparable to FP32 precision. Lastly, we present a fault-tolerant dataflow using redundant RNS error-correcting codes to protect the computation against noise and errors inherent within an analog accelerator.