SPOTS: An Accelerator for Sparse Convolutional Networks Leveraging Systolic General Matrix-Matrix Multiplication

TL;DR

SPOTS is a hardware accelerator for sparse CNNs that uses a novel IM2COL design and a tall systolic array to improve speed and energy efficiency, handling various CNN parameters effectively.

Contribution

The paper introduces a new hardware design combining IM2COL with a systolic GEMM array, optimized for sparsity and flexibility across CNN architectures.

Findings

SPOTS is 1.74X faster than Eyeriss.

It achieves 78X and 12X energy efficiency improvements over CPU and GPU.

The design effectively handles sparsity in both input features and weights.

Abstract

This paper proposes a new hardware accelerator for sparse convolutional neural networks (CNNs) by building a hardware unit to perform the Image to Column (IM2COL) transformation of the input feature map coupled with a systolic array-based general matrix-matrix multiplication (GEMM) unit. Our design carefully overlaps the IM2COL transformation with the GEMM computation to maximize parallelism. We propose a novel design for the IM2COL unit that uses a set of distributed local memories connected by a ring network, which improves energy efficiency and latency by streaming the input feature map only once. We propose a tall systolic array for the GEMM unit while also providing the ability to organize it as multiple small GEMM units, which enables our design to handle a wide range of CNNs and their parameters. Further, our design improves performance by effectively mapping the sparse data to the hardware units by utilizing sparsity in both input feature maps and weights. Our prototype, SPOTS, is on average 1.74X faster than Eyeriss. It is also 78X, and 12X more energy-efficient when compared to CPU and GPU implementations, respectively.