Anatomy Of High-Performance Deep Learning Convolutions On SIMD Architectures

TL;DR

This paper presents high-performance, JIT-optimized direct convolution kernels for x86 CPUs, achieving near-peak efficiency and high throughput in deep learning tasks, especially on multi-node CPU systems.

Contribution

It introduces a novel JIT-based direct convolution implementation for x86 architectures, demonstrating high efficiency and integration into multi-node execution models.

Findings

Near-theoretical peak performance on CPUs

High image-throughput in deep learning tasks

Effective multi-node CPU execution

Abstract

Convolution layers are prevalent in many classes of deep neural networks, including Convolutional Neural Networks (CNNs) which provide state-of-the-art results for tasks like image recognition, neural machine translation and speech recognition. The computationally expensive nature of a convolution operation has led to the proliferation of implementations including matrix-matrix multiplication formulation, and direct convolution primarily targeting GPUs. In this paper, we introduce direct convolution kernels for x86 architectures, in particular for Xeon and XeonPhi systems, which are implemented via a dynamic compilation approach. Our JIT-based implementation shows close to theoretical peak performance, depending on the setting and the CPU architecture at hand. We additionally demonstrate how these JIT-optimized kernels can be integrated into a lightweight multi-node graph execution model. This illustrates that single- and multi-node runs yield high efficiencies and high image-throughputs when executing state-of-the-art image recognition tasks on CPUs.