Minimum Energy Quantized Neural Networks

TL;DR

This paper presents a method to optimize quantized neural networks for minimum energy consumption by analyzing the trade-offs between precision, network architecture, and energy efficiency across different hardware platforms.

Contribution

It introduces a comprehensive energy model for QNN inference and identifies the most energy-efficient bit-widths, such as BinaryNets and int4, outperforming higher precision networks at iso-accuracy.

Findings

BinaryNets and int4 networks are the most energy-efficient at iso-accuracy.

Energy consumption varies significantly with the number of bits used.

Deeper and wider architectures are needed for lower-precision networks to maintain accuracy.

Abstract

This work targets the automated minimum-energy optimization of Quantized Neural Networks (QNNs) - networks using low precision weights and activations. These networks are trained from scratch at an arbitrary fixed point precision. At iso-accuracy, QNNs using fewer bits require deeper and wider network architectures than networks using higher precision operators, while they require less complex arithmetic and less bits per weights. This fundamental trade-off is analyzed and quantified to find the minimum energy QNN for any benchmark and hence optimize energy-efficiency. To this end, the energy consumption of inference is modeled for a generic hardware platform. This allows drawing several conclusions across different benchmarks. First, energy consumption varies orders of magnitude at iso-accuracy depending on the number of bits used in the QNN. Second, in a typical system, BinaryNets or int4 implementations lead to the minimum energy solution, outperforming int8 networks up to 2-10x at iso-accuracy. All code used for QNN training is available from https://github.com/BertMoons.