A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

TL;DR

This paper presents a novel FPGA architecture and a systematic design methodology for efficiently implementing deconvolutional neural networks, enabling high-performance generative inference for computer vision tasks.

Contribution

It introduces an FPGA-based deconvolutional network accelerator with a three-step optimization process, addressing complex memory access patterns and supporting convolution as well.

Findings

Achieved a peak performance density of 0.012 GOPs/DSP on FPGA.

Successfully mapped GAN-trained DCNNs for generative inference.

Demonstrated effectiveness on a Xilinx Zynq-7000 FPGA.

Abstract

In recent years deep learning algorithms have shown extremely high performance on machine learning tasks such as image classification and speech recognition. In support of such applications, various FPGA accelerator architectures have been proposed for convolutional neural networks (CNNs) that enable high performance for classification tasks at lower power than CPU and GPU processors. However, to date, there has been little research on the use of FPGA implementations of deconvolutional neural networks (DCNNs). DCNNs, also known as generative CNNs, encode high-dimensional probability distributions and have been widely used for computer vision applications such as scene completion, scene segmentation, image creation, image denoising, and super-resolution imaging. We propose an FPGA architecture for deconvolutional networks built around an accelerator which effectively handles the complex memory access patterns needed to perform strided deconvolutions, and that supports convolution as well. We also develop a three-step design optimization method that systematically exploits statistical analysis, design space exploration and VLSI optimization. To verify our FPGA deconvolutional accelerator design methodology we train DCNNs offline on two representative datasets using the generative adversarial network method (GAN) run on Tensorflow, and then map these DCNNs to an FPGA DCNN-plus-accelerator implementation to perform generative inference on a Xilinx Zynq-7000 FPGA. Our DCNN implementation achieves a peak performance density of 0.012 GOPs/DSP.