Accelerating Generative Neural Networks on Unmodified Deep Learning Processors -- A Software Approach

TL;DR

This paper introduces a software-based method to accelerate deconvolution in generative neural networks on existing deep learning hardware, achieving significant speedups and energy savings without hardware modifications.

Contribution

It proposes a novel software technique that reorganizes deconvolution computation, enabling legacy processors to perform deconvolution as standard convolution, thus accelerating generative neural networks.

Findings

Achieves 2.41x - 4.34x performance speedup

Reduces energy consumption by 27.7% - 54.5%

Effective on off-the-shelf deep learning processors

Abstract

Generative neural network is a new category of neural networks and it has been widely utilized in applications such as content generation, unsupervised learning, segmentation and pose estimation. It typically involves massive computing-intensive deconvolution operations that cannot be fitted to conventional neural network processors directly. However, prior works mainly investigated specialized hardware architectures through intensive hardware modifications to the existing deep learning processors to accelerate deconvolution together with the convolution. In contrast, this work proposes a novel deconvolution implementation with a software approach and enables fast and efficient deconvolution execution on the legacy deep learning processors. Our proposed method reorganizes the computation of deconvolution and allows the deep learning processors to treat it as the standard convolution by splitting the original deconvolution filters into multiple small filters. Compared to prior acceleration schemes, the implemented acceleration scheme achieves 2.41x - 4.34x performance speedup and reduces the energy consumption by 27.7% - 54.5% on a set of realistic benchmarks. In addition, we also applied the deconvolution computing approach to the off-the-shelf commodity deep learning processors. The performance of deconvolution also exhibits significant performance speedup over prior deconvolution implementations.