Deep Neural Network inference with reduced word length

TL;DR

This paper introduces a method to perform deep neural network inference using low-precision integer arithmetic, significantly reducing computational complexity and hardware requirements while maintaining accuracy.

Contribution

It presents a novel approach to evaluate float32-trained DNNs with 2-3 bit integer arithmetic, enabling efficient hardware implementation.

Findings

DNNs trained with float32 can be evaluated with 2-bit integer arithmetic.

Using 3-bit integer arithmetic with binary shift and clipping maintains performance.

The method enables efficient DNN inference on dedicated hardware.

Abstract

Deep neural networks (DNN) are powerful models for many pattern recognition tasks, yet their high computational complexity and memory requirement limit them to applications on high-performance computing platforms. In this paper, we propose a new method to evaluate DNNs trained with 32bit floating point (float32) accuracy using only low precision integer arithmetics in combination with binary shift and clipping operations. Because hardware implementation of these operations is much simpler than high precision floating point calculation, our method can be used for an efficient DNN inference on dedicated hardware. In experiments on MNIST, we demonstrate that DNNs trained with float32 can be evaluated using a combination of 2bit integer arithmetics and a few float32 calculations in each layer or only 3bit integer arithmetics in combination with binary shift and clipping without significant performance degradation.