Accelerate Three-Dimensional Generative Adversarial Networks Using Fast Algorithm

TL;DR

This paper introduces a fast algorithm (F3DC) for 3D deconvolution in 3D-GANs, significantly reducing computation and memory use, and presents a hardware architecture that boosts efficiency and throughput.

Contribution

The paper proposes a novel fast algorithm for 3D deconvolution that reduces multiplications and memory requirements, and designs a hardware architecture to accelerate 3D-GAN processing.

Findings

Achieves up to 1700 GOPS throughput on FPGA.

Provides 4x computational efficiency improvement over prior methods.

Reduces multiplications and memory use in 3D deconvolution.

Abstract

Three-dimensional generative adversarial networks (3D-GAN) have attracted widespread attention in three-dimension (3D) visual tasks. 3D deconvolution (DeConv), as an important computation of 3D-GAN, significantly increases computational complexity compared with 2D DeConv. 3D DeConv has become a bottleneck for the acceleration of 3D-GAN. Previous accelerators suffer from several problems, such as large memory requirements and resource underutilization. To handle the above issues, a fast algorithm for 3D DeConv (F3DC) is proposed in this paper. F3DC applies a fast algorithm to reduce the number of multiplications and achieves a significant algorithmic strength reduction. Besides, F3DC removes the extra memory requirement for overlapped partial sums and avoids computational imbalance to fully utilize resources. Moreover, we design an F3DC-based hardware architecture, which consists of four fast processing units (FPUs). Each FPU includes a pre-process module, a EWMM module and a post-process module for F3DC transformation. By implementing our design on the Xilinx VC709 platform for 3D-GAN, we achieve a throughput up to 1700 GOPS and 4$\times$ computational efficiency improvement compared with prior works.