Semi-Streaming Architecture: A New Design Paradigm for CNN Implementation on FPGAs

TL;DR

This paper introduces a semi-streaming FPGA architecture for CNNs that combines layer-specific processing engines to improve efficiency and flexibility, demonstrated with an 8-bit MobileNetV2 implementation.

Contribution

It proposes a novel semi-streaming design paradigm that integrates specialized engines for different CNN layers, enhancing resource utilization and performance.

Findings

Achieved up to 89.6 GOp/s throughput for certain layers.

Energy efficiency of 5.32 GOp/s/W at 100MHz.

Implemented a flexible, layer-specific FPGA CNN accelerator.

Abstract

The recent research advances in deep learning have led to the development of small and powerful Convolutional Neural Network (CNN) architectures. Meanwhile Field Programmable Gate Arrays (FPGAs) has become a popular hardware target choice for their deployment, splitting into two main implementation categories: streaming hardware architectures and single computation engine design approaches. The streaming hardware architectures generally require implementing every layer as a discrete processing unit, and are suitable for smaller software models that could fit in their unfolded versions into resource-constrained targets. On the other hand, single computation engines can be scaled to fit into a device to execute CNN models of different sizes and complexities, however, the achievable performance of one-size-fits-all implementations may vary across CNNs with different workload attributes leading to inefficient utilization of hardware resources. By combing the advantages of both of the above methods, this work proposes a new design paradigm called semi-streaming architecture, where layerspecialized configurable engines are used for network realization. As a proof of concept this paper presents a set of five layerspecialized configurable processing engines for implementing 8-bit quantized MobilenevV2 CNN model. The engines are chained to partially preserve data streaming and tuned individually to efficiently process specific types of layers: normalized addition of residuals, depthwise, pointwise (expansion and projection), and standard 2D convolution layers capable of delivering 5.4GOp/s, 16GOp/s, 27.2GOp/s, 27.2GOp/s and 89.6GOp/s, respectively, with the overall energy efficiency of 5.32GOp/s/W at a 100MHz system clock, requiring total power of 6.2W on a XCZU7EV SoC FPGA.