Im2win: Memory Efficient Convolution On SIMD Architectures

TL;DR

This paper introduces im2win, a memory-efficient data transformation algorithm for convolution in neural networks that reduces memory footprint and improves performance on SIMD architectures.

Contribution

The paper presents a novel im2win algorithm that refactors data transformation for convolution, significantly reducing memory usage and increasing computational speed.

Findings

Reduces memory overhead by 41.6% compared to im2col-based methods.

Achieves up to 3.6× speedup over im2col-based convolution.

Achieves up to 5.3× speedup over not using data transformation.

Abstract

Convolution is the most expensive operation among neural network operations, thus its performance is critical to the overall performance of neural networks. Commonly used convolution approaches, including general matrix multiplication (GEMM)-based convolution and direct convolution, rely on im2col for data transformation or do not use data transformation at all, respectively. However, the im2col data transformation can lead to at least 2$\times$ memory footprint compared to not using data transformation at all, thus limiting the size of neural network models running on memory-limited systems. Meanwhile, not using data transformation usually performs poorly due to nonconsecutive memory access although it consumes less memory. To solve those problems, we propose a new memory-efficient data transformation algorithm, called im2win. This algorithm refactorizes a row of square or rectangle dot product windows of the input image and flattens unique elements within these windows into a row in the output tensor, which enables consecutive memory access and data reuse, and thus greatly reduces the memory overhead. Furthermore, we propose a high-performance im2win-based convolution algorithm with various optimizations, including vectorization, loop reordering, etc. Our experimental results show that our algorithm reduces the memory overhead by average to 41.6% compared to the PyTorch's convolution implementation based on im2col, and achieves average to 3.6$\times$ and 5.3$\times$ speedup in performance compared to the im2col-based convolution and not using data transformation, respectively.