GrateTile: Efficient Sparse Tensor Tiling for CNN Processing

TL;DR

GrateTile introduces a hardware-friendly sparse tensor tiling scheme that significantly reduces DRAM bandwidth in CNN accelerators by efficiently storing and accessing sparse feature maps with minimal indexing overhead.

Contribution

It presents a novel sparse tensor tiling method that enables efficient on-the-fly decompression and access, reducing bandwidth and storage overhead in CNN processing.

Findings

55% average DRAM bandwidth reduction

Only 0.6% feature map size used for indexing

Compatible with modern CNN accelerators

Abstract

We propose GrateTile, an efficient, hardwarefriendly data storage scheme for sparse CNN feature maps (activations). It divides data into uneven-sized subtensors and, with small indexing overhead, stores them in a compressed yet randomly accessible format. This design enables modern CNN accelerators to fetch and decompressed sub-tensors on-the-fly in a tiled processing manner. GrateTile is suitable for architectures that favor aligned, coalesced data access, and only requires minimal changes to the overall architectural design. We simulate GrateTile with state-of-the-art CNNs and show an average of 55% DRAM bandwidth reduction while using only 0.6% of feature map size for indexing storage.