Efficient Edge AI: Deploying Convolutional Neural Networks on FPGA with the Gemmini Accelerator

TL;DR

This paper presents an end-to-end workflow for deploying CNNs on FPGAs using a modified Gemmini accelerator, achieving real-time performance and high energy efficiency for edge AI applications.

Contribution

It introduces a customized FPGA deployment process for CNNs with open source tools, demonstrating improved performance and energy efficiency over existing solutions.

Findings

Achieved real-time YOLOv7 deployment on FPGA

Energy efficiency of 36.5 GOP/s/W

Outperforms other FPGA implementations

Abstract

The growing concerns regarding energy consumption and privacy have prompted the development of AI solutions deployable on the edge, circumventing the substantial CO2 emissions associated with cloud servers and mitigating risks related to sharing sensitive data. But deploying Convolutional Neural Networks (CNNs) on non-off-the-shelf edge devices remains a complex and labor-intensive task. In this paper, we present and end-to-end workflow for deployment of CNNs on Field Programmable Gate Arrays (FPGAs) using the Gemmini accelerator, which we modified for efficient implementation on FPGAs. We describe how we leverage the use of open source software on each optimization step of the deployment process, the customizations we added to them and its impact on the final system's performance. We were able to achieve real-time performance by deploying a YOLOv7 model on a Xilinx ZCU102 FPGA with an energy efficiency of 36.5 GOP/s/W. Our FPGA-based solution demonstrates superior power efficiency compared with other embedded hardware devices, and even outperforms other FPGA reference implementations. Finally, we present how this kind of solution can be integrated into a wider system, by testing our proposed platform in a traffic monitoring scenario.