Bit-balance: Model-Hardware Co-design for Accelerating NNs by Exploiting Bit-level Sparsity

TL;DR

This paper introduces Bit-balance, a co-designed model-hardware approach leveraging bit-level sparsity in neural networks to enhance resource and energy efficiency through balanced workloads and adaptive bitwidth computation.

Contribution

It proposes a novel bit-sparsity quantization method and a sparse bit-serial architecture for improved neural network acceleration.

Findings

Achieves 1.8x~2.7x energy efficiency over existing accelerators.

Supports multiple neural network architectures with high frame rates.

Maintains accuracy with minimal impact from bit sparsity constraints.

Abstract

Bit-serial architectures can handle Neural Networks (NNs) with different weight precisions, achieving higher resource efficiency compared with bit-parallel architectures. Besides, the weights contain abundant zero bits owing to the fault tolerance of NNs, indicating that bit sparsity of NNs can be further exploited for performance improvement. However, the irregular proportion of zero bits in each weight causes imbalanced workloads in the Processing Element (PE) array, which degrades performance or induces overhead for sparse processing. Thus, this paper proposed a bit-sparsity quantization method to maintain the bit sparsity ratio of each weight to no more than a certain value for balancing workloads, with little accuracy loss. Then, we co-designed a sparse bit-serial architecture, called Bit-balance, to improve overall performance, supporting weight-bit sparsity and adaptive bitwidth computation. The whole design was implemented with 65nm technology at 1 GHz and performs at 326-, 30-, 56-, and 218-frame/s for AlexNet, VGG-16, ResNet-50, and GoogleNet respectively. Compared with sparse bit-serial accelerator, Bitlet, Bit-balance achieves 1.8x~2.7x energy efficiency (frame/J) and 2.1x~3.7x resource efficiency (frame/mm2).