Towards Accurate and High-Speed Spiking Neuromorphic Systems with Data Quantization-Aware Deep Networks

TL;DR

This paper proposes a data quantization-aware deep network implementation on spiking neuromorphic systems, achieving high accuracy, speed, and energy efficiency by using fixed-point weights and signals.

Contribution

It introduces a novel approach of representing DNNs with fixed integer signals and fixed-point weights for neuromorphic deployment, reducing accuracy loss.

Findings

Accuracy loss within 0.02% on MNIST and 2.3% on CIFAR-10.

Over 9.8x speedup compared to 8-bit DNNs.

89.1% energy savings and 30% area reduction.

Abstract

Deep Neural Networks (DNNs) have gained immense success in cognitive applications and greatly pushed today's artificial intelligence forward. The biggest challenge in executing DNNs is their extremely data-extensive computations. The computing efficiency in speed and energy is constrained when traditional computing platforms are employed in such computational hungry executions. Spiking neuromorphic computing (SNC) has been widely investigated in deep networks implementation own to their high efficiency in computation and communication. However, weights and signals of DNNs are required to be quantized when deploying the DNNs on the SNC, which results in unacceptable accuracy loss. %However, the system accuracy is limited by quantizing data directly in deep networks deployment. Previous works mainly focus on weights discretize while inter-layer signals are mainly neglected. In this work, we propose to represent DNNs with fixed integer inter-layer signals and fixed-point weights while holding good accuracy. We implement the proposed DNNs on the memristor-based SNC system as a deployment example. With 4-bit data representation, our results show that the accuracy loss can be controlled within 0.02% (2.3%) on MNIST (CIFAR-10). Compared with the 8-bit dynamic fixed-point DNNs, our system can achieve more than 9.8x speedup, 89.1% energy saving, and 30% area saving.