Memory-Efficient CNN Accelerator Based on Interlayer Feature Map Compression

TL;DR

This paper introduces a hardware accelerator for CNNs that uses frequency domain transformation and sparse compression to significantly reduce memory requirements and bandwidth, enabling efficient real-time processing on embedded devices.

Contribution

It presents a novel interlayer feature map compression technique integrated into a CNN accelerator, combining DCT, quantization, and sparse matrix compression for the first time.

Findings

Achieves 403GOPS peak throughput on FPGA

Reduces interlayer feature map size by up to 3.3x

Maintains minimal delay with integrated compression and processing

Abstract

Existing deep convolutional neural networks (CNNs) generate massive interlayer feature data during network inference. To maintain real-time processing in embedded systems, large on-chip memory is required to buffer the interlayer feature maps. In this paper, we propose an efficient hardware accelerator with an interlayer feature compression technique to significantly reduce the required on-chip memory size and off-chip memory access bandwidth. The accelerator compresses interlayer feature maps through transforming the stored data into frequency domain using hardware-implemented 8x8 discrete cosine transform (DCT). The high-frequency components are removed after the DCT through quantization. Sparse matrix compression is utilized to further compress the interlayer feature maps. The on-chip memory allocation scheme is designed to support dynamic configuration of the feature map buffer size and scratch pad size according to different network-layer requirements. The hardware accelerator combines compression, decompression, and CNN acceleration into one computing stream, achieving minimal compressing and processing delay. A prototype accelerator is implemented on an FPGA platform and also synthesized in TSMC 28-nm COMS technology. It achieves 403GOPS peak throughput and 1.4x~3.3x interlayer feature map reduction by adding light hardware area overhead, making it a promising hardware accelerator for intelligent IoT devices.