DSLOT-NN: Digit-Serial Left-to-Right Neural Network Accelerator

TL;DR

DSLOT-NN introduces a digit-serial, left-to-right neural network accelerator that significantly reduces power consumption and improves efficiency by using online arithmetic operators and early termination of ineffective convolutions.

Contribution

It proposes a novel digit-serial left-to-right processing technique with online operators for DNN inference acceleration, enabling power savings and dynamic precision tuning.

Findings

50% higher OPS per watt compared to state-of-the-art

Shorter cycle time and lower power consumption

Effective early termination of ineffective convolutions

Abstract

We propose a Digit-Serial Left-tO-righT (DSLOT) arithmetic based processing technique called DSLOT-NN with aim to accelerate inference of the convolution operation in the deep neural networks (DNNs). The proposed work has the ability to assess and terminate the ineffective convolutions which results in massive power and energy savings. The processing engine is comprised of low-latency most-significant-digit-first (MSDF) (also called online) multipliers and adders that processes data from left-to-right, allowing the execution of subsequent operations in digit-pipelined manner. Use of online operators eliminates the need for the development of complex mechanism of identifying the negative activation, as the output with highest weight value is generated first, and the sign of the result can be identified as soon as first non-zero digit is generated. The precision of the online operators can be tuned at run-time, making them extremely useful in situations where accuracy can be compromised for power and energy savings. The proposed design has been implemented on Xilinx Virtex-7 FPGA and is compared with state-of-the-art Stripes on various performance metrics. The results show the proposed design presents power savings, has shorter cycle time, and approximately 50% higher OPS per watt.