FPGA-Based CNN Inference Accelerator Synthesized from Multi-Threaded C Software

TL;DR

This paper presents an FPGA-based CNN inference accelerator synthesized from multi-threaded C software, leveraging high-level synthesis to convert software parallelism into hardware, achieving high performance on VGG-16.

Contribution

It introduces a novel approach to synthesize CNN inference accelerators from multi-threaded C code using high-level synthesis, including zero-weight-skipping and reduced precision techniques.

Findings

Peak performance of 138 effective GOPS on VGG-16

Successful synthesis of convolution, pooling, and padding in FPGA hardware

System combines FPGA accelerator with embedded ARM processor

Abstract

A deep-learning inference accelerator is synthesized from a C-language software program parallelized with Pthreads. The software implementation uses the well-known producer/consumer model with parallel threads interconnected by FIFO queues. The LegUp high-level synthesis (HLS) tool synthesizes threads into parallel FPGA hardware, translating software parallelism into spatial parallelism. A complete system is generated where convolution, pooling and padding are realized in the synthesized accelerator, with remaining tasks executing on an embedded ARM processor. The accelerator incorporates reduced precision, and a novel approach for zero-weight-skipping in convolution. On a mid-sized Intel Arria 10 SoC FPGA, peak performance on VGG-16 is 138 effective GOPS.