Overview of FPGA deep learning acceleration based on convolutional neural network

TL;DR

This paper reviews FPGA-based acceleration of convolutional neural networks, discussing theories, algorithms, application scenarios, and challenges like resource under-utilization to enhance deep learning performance.

Contribution

It provides a comprehensive overview of FPGA acceleration techniques for CNNs, highlighting existing applications and identifying resource utilization issues.

Findings

FPGA accelerators are widely used for CNNs in visual tasks.

Many accelerators suffer from under-utilization of logic resources or memory bandwidth.

Addressing resource under-utilization can improve FPGA performance for CNNs.

Abstract

In recent years, deep learning has become more and more mature, and as a commonly used algorithm in deep learning, convolutional neural networks have been widely used in various visual tasks. In the past, research based on deep learning algorithms mainly relied on hardware such as GPUs and CPUs. However, with the increasing development of FPGAs, both field programmable logic gate arrays, it has become the main implementation hardware platform that combines various neural network deep learning algorithms This article is a review article, which mainly introduces the related theories and algorithms of convolution. It summarizes the application scenarios of several existing FPGA technologies based on convolutional neural networks, and mainly introduces the application of accelerators. At the same time, it summarizes some accelerators' under-utilization of logic resources or under-utilization of memory bandwidth, so that they can't get the best performance.