Unified Kernel-Segregated Transpose Convolution Operation

TL;DR

This paper introduces a unified kernel segregation method for transpose convolution that reduces memory and computational costs, achieving over 2x speedup and significant memory savings in deep learning models.

Contribution

The paper proposes a novel unified kernel segregation approach that optimizes transpose convolution operations, improving speed and memory efficiency compared to existing methods.

Findings

Achieves 2.03x to 3.89x speedup on GPUs and CPUs.

Reduces memory usage by up to 35 MB in GAN models.

Provides significant acceleration in transpose convolution layers.

Abstract

The optimization of the transpose convolution layer for deep learning applications is achieved with the kernel segregation mechanism. However, kernel segregation has disadvantages, such as computing extra elements to obtain the output feature map with odd dimensions while launching a thread. To mitigate this problem, we introduce a unified kernel segregation approach that limits the usage of memory and computational resources by employing one unified kernel to execute four sub-kernels. The findings reveal that the suggested approach achieves an average computational speedup of 2.03x (3.89x) when tested on specific datasets with an RTX 2070 GPU (Intel Xeon CPU). The ablation study shows an average computational speedup of 3.5x when evaluating the transpose convolution layers from well-known Generative Adversarial Networks (GANs). The implementation of the proposed method for the transpose convolution layers in the EB-GAN model demonstrates significant memory savings of up to 35 MB.