A Survey of FPGA Based Deep Learning Accelerators: Challenges and Opportunities

TL;DR

This paper systematically reviews FPGA-based deep learning accelerators, comparing their designs, performance, and potential, highlighting challenges and opportunities for future research in hardware acceleration.

Contribution

It provides a comprehensive survey of FPGA accelerators for deep learning, analyzing design approaches, device comparisons, and discussing future research directions.

Findings

FPGA accelerators offer advantages over CPU and GPU in specific deep learning tasks.

Design diversity exists for problem-specific and general-purpose FPGA accelerators.

Challenges include balancing performance, power, and flexibility in FPGA-based solutions.

Abstract

With the rapid development of in-depth learning, neural network and deep learning algorithms have been widely used in various fields, e.g., image, video and voice processing. However, the neural network model is getting larger and larger, which is expressed in the calculation of model parameters. Although a wealth of existing efforts on GPU platforms currently used by researchers for improving computing performance, dedicated hardware solutions are essential and emerging to provide advantages over pure software solutions. In this paper, we systematically investigate the neural network accelerator based on FPGA. Specifically, we respectively review the accelerators designed for specific problems, specific algorithms, algorithm features, and general templates. We also compared the design and implementation of the accelerator based on FPGA under different devices and network models and compared it with the versions of CPU and GPU. Finally, we present to discuss the advantages and disadvantages of accelerators on FPGA platforms and to further explore the opportunities for future research.