Sorted Weight Sectioning for Energy-Efficient Unstructured Sparse DNNs on Compute-in-Memory Crossbars

TL;DR

This paper presents sorted weight sectioning (SWS), an algorithm that optimizes weight placement on compute-in-memory crossbars to significantly reduce ADC energy consumption in unstructured sparse DNNs, especially for models like BERT.

Contribution

The paper introduces SWS, a novel weight allocation algorithm that leverages weight distribution and sparsity to minimize ADC energy use in CIM crossbars for DNNs.

Findings

Reduces ADC energy by 89.5% on unstructured sparse BERT models.

Effectively exploits weight distribution and sparsity for energy savings.

Maintains accuracy while significantly decreasing energy consumption.

Abstract

We introduce $\textit{sorted weight sectioning}$ (SWS): a weight allocation algorithm that places sorted deep neural network (DNN) weight sections on bit-sliced compute-in-memory (CIM) crossbars to reduce analog-to-digital converter (ADC) energy consumption. Data conversions are the most energy-intensive process in crossbar operation. SWS effectively reduces this cost leveraging (1) small weights and (2) zero weights (weight sparsity). DNN weights follow bell-shaped distributions, with most weights near zero. Using SWS, we only need low-order crossbar columns for sections with low-magnitude weights. This reduces the quantity and resolution of ADCs used, exponentially decreasing ADC energy costs without significantly degrading DNN accuracy. Unstructured sparsification further sharpens the weight distribution with small accuracy loss. However, it presents challenges in hardware tracking of zeros: we cannot switch zero rows to other layer weights in unsorted crossbars without index matching. SWS efficiently addresses unstructured sparse models using offline remapping of zeros into earlier sections, which reveals full sparsity potential and maximizes energy efficiency. Our method reduces ADC energy use by 89.5% on unstructured sparse BERT models. Overall, this paper introduces a novel algorithm to promote energy-efficient CIM crossbars for unstructured sparse DNN workloads.