Low- and Mixed-Precision Inference Accelerators

TL;DR

This paper reviews hardware accelerators designed for low- and mixed-precision neural network inference, emphasizing their energy efficiency and flexibility for edge computing devices with strict resource constraints.

Contribution

It provides a comprehensive review of design choices and implications of accelerators supporting extremely quantized neural networks for edge inference.

Findings

Accelerators enable energy-efficient neural network inference on edge devices.

Design choices impact flexibility and energy consumption of low-precision accelerators.

Quantization to binary or low-precision improves energy efficiency significantly.

Abstract

With the surging popularity of edge computing, the need to efficiently perform neural network inference on battery-constrained IoT devices has greatly increased. While algorithmic developments enable neural networks to solve increasingly more complex tasks, the deployment of these networks on edge devices can be problematic due to the stringent energy, latency, and memory requirements. One way to alleviate these requirements is by heavily quantizing the neural network, i.e. lowering the precision of the operands. By taking quantization to the extreme, e.g. by using binary values, new opportunities arise to increase the energy efficiency. Several hardware accelerators exploiting the opportunities of low-precision inference have been created, all aiming at enabling neural network inference at the edge. In this chapter, design choices and their implications on the flexibility and energy efficiency of several accelerators supporting extremely quantized networks are reviewed.