# Optimizing Memory Efficiency for Convolution Kernels on Kepler GPUs

## TL;DR

This paper presents optimized convolution kernels for Kepler GPUs that significantly improve memory efficiency and performance, addressing the mismatch between GPU memory architecture and computation data width.

## Contribution

It introduces a general model for memory-data width mismatch and develops two optimized convolution kernels, achieving substantial performance gains over cuDNN.

## Key findings

- Achieved 5.16X performance improvement for the special case.
- Achieved 35.5% average performance improvement for the general case.
- Developed memory access and computation pattern optimizations.

## Abstract

Convolution is a fundamental operation in many applications, such as computer vision, natural language processing, image processing, etc. Recent successes of convolutional neural networks in various deep learning applications put even higher demand on fast convolution. The high computation throughput and memory bandwidth of graphics processing units (GPUs) make GPUs a natural choice for accelerating convolution operations. However, maximally exploiting the available memory bandwidth of GPUs for convolution is a challenging task. This paper introduces a general model to address the mismatch between the memory bank width of GPUs and computation data width of threads. Based on this model, we develop two convolution kernels, one for the general case and the other for a special case with one input channel. By carefully optimizing memory access patterns and computation patterns, we design a communication-optimized kernel for the special case and a communication-reduced kernel for the general case. Experimental data based on implementations on Kepler GPUs show that our kernels achieve 5.16X and 35.5% average performance improvement over the latest cuDNN library, for the special case and the general case, respectively.