Towards Effective Depthwise Convolutions on ARMv8 Architecture

TL;DR

This paper introduces optimized direct implementations of depthwise convolutions on ARMv8, significantly improving performance by reducing memory overhead and enhancing register reuse in CNN computations.

Contribution

The paper presents novel direct algorithms for depthwise convolutions on ARMv8, utilizing implicit padding and register tiling for better efficiency.

Findings

Achieved 4.88x performance improvement over existing direct implementations.

Achieved 16.4x performance improvement over matrix multiplication-based implementations.

Demonstrated effectiveness on two ARMv8 CPUs with open source libraries and Pytorch.

Abstract

Depthwise convolutions are widely used in lightweight convolutional neural networks (CNNs). The performance of depthwise convolutions is mainly bounded by the memory access rather than the arithmetic operations for classic convolutions so that direct algorithms are often more efficient than indirect ones (matrix multiplication-, Winograd-, and FFT-based convolutions) with additional memory accesses. However, the existing direct implementations of depthwise convolutions on ARMv8 architectures feature a bad trade-off between register-level reuse of different tensors, which usually leads to sub-optimal performance. In this paper, we propose new direct implementations of depthwise convolutions by means of implicit padding, register tiling, etc., which contain forward propagation, backward propagation and weight gradient update procedures. Compared to the existing ones, our new implementations can incur much less communication overhead between registers and cache. Experimental results on two ARMv8 CPUs show that our implementations can averagely deliver 4.88x and 16.4x performance improvement over the existing direct ones in open source libraries and matrix multiplications-based ones in Pytorch, respectively.