PalQuant: Accelerating High-precision Networks on Low-precision Accelerators

TL;DR

PalQuant is a novel method that enables high-precision neural networks to run efficiently on low-precision accelerators by learning parallel low-precision representations and using a cyclic shuffle module, improving accuracy and speed.

Contribution

The paper introduces PalQuant, a new approach for approximating high-precision computations with parallel low-precision representations and a cyclic shuffle module, enhancing performance on low-precision hardware.

Findings

PalQuant achieves 0.52% higher accuracy than existing methods.

PalQuant provides a 1.78× inference speedup on a 2-bit accelerator.

It outperforms state-of-the-art quantization methods in accuracy and efficiency.

Abstract

Recently low-precision deep learning accelerators (DLAs) have become popular due to their advantages in chip area and energy consumption, yet the low-precision quantized models on these DLAs bring in severe accuracy degradation. One way to achieve both high accuracy and efficient inference is to deploy high-precision neural networks on low-precision DLAs, which is rarely studied. In this paper, we propose the PArallel Low-precision Quantization (PalQuant) method that approximates high-precision computations via learning parallel low-precision representations from scratch. In addition, we present a novel cyclic shuffle module to boost the cross-group information communication between parallel low-precision groups. Extensive experiments demonstrate that PalQuant has superior performance to state-of-the-art quantization methods in both accuracy and inference speed, e.g., for ResNet-18 network quantization, PalQuant can obtain 0.52\% higher accuracy and 1.78$\times$ speedup simultaneously over their 4-bit counter-part on a state-of-the-art 2-bit accelerator. Code is available at \url{https://github.com/huqinghao/PalQuant}.