A flexible FPGA accelerator for convolutional neural networks

TL;DR

This paper presents a flexible FPGA-based CNN inference accelerator that minimizes off-chip memory access through various reuse strategies, supports high resource utilization without reconfiguration, and integrates with TensorFlow for ease of programming.

Contribution

The paper introduces a novel FPGA CNN accelerator design that exploits multiple reuse strategies, maintains high utilization without reconfiguration, and provides a TensorFlow-compatible software framework.

Findings

Achieves high frequency scaling with increased PEs.

Maintains a significant fraction of theoretical peak performance.

Effectively reduces off-chip memory access in CNN inference.

Abstract

Though CNNs are highly parallel workloads, in the absence of efficient on-chip memory reuse techniques, an accelerator for them quickly becomes memory bound. In this paper, we propose a CNN accelerator design for inference that is able to exploit all forms of reuse available to minimize off-chip memory access while increasing utilization of available resources. The proposed design is composed of cores, each of which contains a one-dimensional array of processing elements. These cores can exploit different types of reuse available in CNN layers of varying shapes without requiring any reconfiguration; in particular, our design minimizes underutilization due to problem sizes that are not perfect multiples of the underlying hardware array dimensions. A major obstacle in the adoption of FPGAs as a platform for CNN inference is the difficulty to program these devices using hardware description languages. Our end goal is to also address this, and we develop preliminary software support via a codesign in order to leverage the accelerator through TensorFlow, a dominant high-level programming model. Our framework takes care of tiling and scheduling of neural network layers and generates necessary low-level commands to execute the CNN. Experimental evaluation on a real system with a PCI-express based Xilinx VC709 board demonstrates the effectiveness of our approach. As a result of an effective interconnection, the design maintains a high frequency when we scale the number of PEs. The sustained performance overall is a good fraction of the accelerator's theoretical peak performance.