Caffeinated FPGAs: FPGA Framework For Convolutional Neural Networks

TL;DR

This paper introduces a modified version of the Caffe framework that supports FPGA implementations of CNNs, enabling flexible, reprogrammable, and efficient neural network processing on FPGA hardware.

Contribution

It presents an FPGA-compatible extension of Caffe, including a Winograd convolution engine, facilitating CNN deployment on FPGA with seamless integration and high performance.

Findings

Achieves 50 GFLOPS on 3x3 convolutions

Supports multiple CNN architectures like AlexNet and VGG

Enables flexible FPGA-based CNN deployment

Abstract

Convolutional Neural Networks (CNNs) have gained significant traction in the field of machine learning, particularly due to their high accuracy in visual recognition. Recent works have pushed the performance of GPU implementations of CNNs to significantly improve their classification and training times. With these improvements, many frameworks have become available for implementing CNNs on both CPUs and GPUs, with no support for FPGA implementations. In this work we present a modified version of the popular CNN framework Caffe, with FPGA support. This allows for classification using CNN models and specialized FPGA implementations with the flexibility of reprogramming the device when necessary, seamless memory transactions between host and device, simple-to-use test benches, and the ability to create pipelined layer implementations. To validate the framework, we use the Xilinx SDAccel environment to implement an FPGA-based Winograd convolution engine and show that the FPGA layer can be used alongside other layers running on a host processor to run several popular CNNs (AlexNet, GoogleNet, VGG A, Overfeat). The results show that our framework achieves 50 GFLOPS across 3x3 convolutions in the benchmarks. This is achieved within a practical framework, which will aid in future development of FPGA-based CNNs.