A Bit Level Weight Reordering Strategy Based on Column Similarity to Explore Weight Sparsity in RRAM-based NN Accelerator

TL;DR

This paper introduces a bit-level weight reordering strategy for RRAM-based neural network accelerators that enhances weight sparsity utilization, leading to significant performance and energy efficiency improvements.

Contribution

The paper proposes a novel bit reordering method that exploits bit similarity and sparsity to improve weight mapping in RRAM-based accelerators, enabling better integration of CIM and sparsity techniques.

Findings

61.24% average performance improvement

1.51x-2.52x energy savings

Effective weight compression with minimal overhead

Abstract

Compute-in-Memory (CIM) and weight sparsity are two effective techniques to reduce data movement during Neural Network (NN) inference. However, they can hardly be employed in the same accelerator simultaneously because CIM requires structural compute patterns which are disrupted in sparse NNs. In this paper, we partially solve this issue by proposing a bit level weight reordering strategy which can realize compact mapping of sparse NN weight matrices onto Resistive Random Access Memory (RRAM) based NN Accelerators (RRAM-Acc). In specific, when weights are mapped to RRAM crossbars in a binary complement manner, we can observe that, which can also be mathematically proven, bit-level sparsity and similarity commonly exist in the crossbars. The bit reordering method treats bit sparsity as a special case of bit similarity, reserve only one column in a pair of columns that have identical bit values, and then map the compressed weight matrices into Operation Units (OU). The performance of our design is evaluated with typical NNs. Simulation results show a 61.24% average performance improvement and 1.51x-2.52x energy savings under different sparsity ratios, with only slight overhead compared to the state-of-the-art design.