Towards Design Methodology of Efficient Fast Algorithms for Accelerating Generative Adversarial Networks on FPGAs

TL;DR

This paper introduces a novel FPGA accelerator for GANs that combines Winograd minimal filtering with optimized dataflow and architecture design, significantly improving efficiency and speed over existing methods.

Contribution

It proposes a new Winograd-based DeConv algorithm, a restructured dataflow, and an optimized architecture for FPGA implementation, enhancing GAN acceleration.

Findings

Achieves up to 8.38x speedup over state-of-the-art DeConv accelerators.

Effectively reduces computational complexity using Winograd minimal filtering.

Improves resource utilization through a novel dataflow and filter layout.

Abstract

Generative adversarial networks (GANs) have shown excellent performance in image and speech applications. GANs create impressive data primarily through a new type of operator called deconvolution (DeConv) or transposed convolution (Conv). To implement the DeConv layer in hardware, the state-of-the-art accelerator reduces the high computational complexity via the DeConv-to-Conv conversion and achieves the same results. However, there is a problem that the number of filters increases due to this conversion. Recently, Winograd minimal filtering has been recognized as an effective solution to improve the arithmetic complexity and resource efficiency of the Conv layer. In this paper, we propose an efficient Winograd DeConv accelerator that combines these two orthogonal approaches on FPGAs. Firstly, we introduce a new class of fast algorithm for DeConv layers using Winograd minimal filtering. Since there are regular sparse patterns in Winograd filters, we further amortize the computational complexity by skipping zero weights. Secondly, we propose a new dataflow to prevent resource underutilization by reorganizing the filter layout in the Winograd domain. Finally, we propose an efficient architecture for implementing Winograd DeConv by designing the line buffer and exploring the design space. Experimental results on various GANs show that our accelerator achieves up to 1.78x~8.38x speedup over the state-of-the-art DeConv accelerators.