Design of High-Throughput Mixed-Precision CNN Accelerators on FPGA

TL;DR

This paper presents a comprehensive methodology for designing high-throughput, mixed-precision CNN accelerators on FPGA, optimizing accuracy and efficiency through layer-wise quantization and resource-aware exploration.

Contribution

It introduces a holistic, multi-level exploration approach for FPGA-based mixed-precision CNN accelerators, enabling efficient layer-wise and channel-wise quantization with significant resource savings.

Findings

Achieves 245 fps with 87.48% Top-5 accuracy on ResNet-18

Reaches 92.9% Top-5 accuracy at 1.13 TOPs/s on ResNet-152

Reduces parameter memory footprint by up to 9.4x

Abstract

Convolutional Neural Networks (CNNs) reach high accuracies in various application domains, but require large amounts of computation and incur costly data movements. One method to decrease these costs while trading accuracy is weight and/or activation word-length reduction. Thereby, layer-wise mixed-precision quantization allows for more efficient results while inflating the design space. In this work, we present an in-depth quantitative methodology to efficiently explore the design space considering the limited hardware resources of a given FPGA. Our holistic exploration approach vertically traverses the various design entry levels from the architectural down to the logic level, and laterally covers optimization from processing elements to dataflow for an efficient mixed-precision CNN accelerator. Our resulting hardware accelerators implement truly mixed-precision operations that enable efficient execution of layer-wise and channel-wise quantized CNNs. Mapping feed-forward and identity-shortcut-connection mixed-precision CNNs result in competitive accuracy-throughout trade-offs: 245 frames/s with 87.48% Top-5 accuracy for ResNet-18 and 92.9% Top-5 accuracy with 1.13 TOps/s for ResNet-152, respectively. Thereby, the required memory footprint for parameters is reduced by 4.9x and 9.4x compared to the respective floating-point baseline.