Caffe Barista: Brewing Caffe with FPGAs in the Training Loop

TL;DR

This paper introduces Barista, an automated toolflow that enables seamless integration of FPGAs into CNN training within Caffe, facilitating rapid and versatile deployment of hardware and algorithms for power-efficient deep learning training.

Contribution

Barista is the first tool that allows easy prototyping and deployment of FPGA-based CNN training within a popular deep learning framework, advancing research in power-efficient training.

Findings

Enables FPGA integration into CNN training workflows.

Supports rapid prototyping of hardware and algorithmic techniques.

Facilitates research into power-efficient deep learning training.

Abstract

As the complexity of deep learning (DL) models increases, their compute requirements increase accordingly. Deploying a Convolutional Neural Network (CNN) involves two phases: training and inference. With the inference task typically taking place on resource-constrained devices, a lot of research has explored the field of low-power inference on custom hardware accelerators. On the other hand, training is both more compute- and memory-intensive and is primarily performed on power-hungry GPUs in large-scale data centres. CNN training on FPGAs is a nascent field of research. This is primarily due to the lack of tools to easily prototype and deploy various hardware and/or algorithmic techniques for power-efficient CNN training. This work presents Barista, an automated toolflow that provides seamless integration of FPGAs into the training of CNNs within the popular deep learning framework Caffe. To the best of our knowledge, this is the only tool that allows for such versatile and rapid deployment of hardware and algorithms for the FPGA-based training of CNNs, providing the necessary infrastructure for further research and development.